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MEASURING  LEARNING  AND  DEVELOPMENT  IN  CROSS-CULTURAL  COMPETENCE 


EXECUTIVE  SUMMARY 


Research  Requirement: 

For  Soldiers  in  the  contemporary  operational  environment,  performance  beyond  typical 
military  technical  and  tactical  skills  is  increasingly  required.  For  example,  Soldiers  must 
effectively  interact  with,  engage,  and  persuade  local  populations  in  foreign  countries  to  achieve 
mission  objectives.  While  many  cultural  training  initiatives  are  currently  operational,  the  lack  of 
a  sound  method  for  assessing  cross-cultural  competence  (3C)  in  Soldiers  leaves  their 
effectiveness  uncertain.  To  ensure  that  Soldiers  are  adequately  prepared  for  the  challenges  of 
active  duty  in  unfamiliar  cultures,  3C  must  be  clearly  defined  and  assessed  at  the  individual 
level.  With  these  objectives  in  mind,  this  Small  Business  Innovation  Research  (SBIR)  research 
effort  had  two  main  thrusts:  (1)  model  the  development  and  components  of  3C  at  the  tactical 
level,  and  (2)  develop  a  battery  of  metrics  that  reliably  and  validly  assesses  Soldiers’  3C. 

Procedure: 

In  Phase  1  of  the  current  effort,  the  research  team  leveraged  cognitive  task  analysis 
methodologies  with  more  than  400  Soldiers  to  create  a  developmental  model  of  the  knowledge, 
skills,  aptitudes,  and  abilities  (KSAAs)  that  comprise  Anny  cross-cultural  competence.  Data 
were  compiled  and  analyzed  with  a  focus  on  identifying  patterns  and  frequencies  in  the  presence 
and/or  mention  of  specific  KSAAs  and  an  initial  set  of  cognitive,  affective,  and  behavioral 
KSAAs  were  identified  as  representing  an  overall  3C  construct.  Within  this  preliminary  3C 
model,  levels  of  competence  were  also  identified  and  represented  in  order  of  increasing 
competence  (e.g.,  precompetent  to  advanced). 

The  goal  of  Phase  II  was  to  refine  the  Phase  I  model  and  develop  a  battery  of  metrics  for 
assessing  Soldiers’  3C.  After  revised  the  initial  set  of  KSAAs  to  focus  on  those  that  were  most 
relevant  to  cross-cultural  competence  development,  the  research  team  generated  a  self-report 
measure  with  items  developed  through  a  review  of  the  relevant  literature  as  well  as  team  and 
critical  incident  interviews  with  Soldiers.  Questions  prompted  participants  to  report  on  general 
behaviors  and  preferences  as  well  as  hypothetical  or  future  deployment  situations.  To  address 
the  limitations  of  a  self-report  assessment  approach,  situational  judgment  tests  (SJTs)  and 
cultural  vignettes  were  developed  and  added  to  the  larger  measurement  battery. 

Soldiers  and  ROTC  cadets  completed  the  3C  assessment  battery  comprised  of  a 
demographics  questionnaire,  self-report,  SJTs,  and  vignettes.  However,  several  modifications 
were  made  to  the  items  administered  to  cadets  to  better  relate  to  the  cadets’  experiences  in  a  two- 
week  summer  cross-cultural  immersion  program.  Cadets  completed  a  pre-test  version  of  a  cross- 
cultural  competence  assessment  battery,  traveled  abroad  for  approximately  two  weeks,  and 
immediately  completed  a  post-test  version  of  the  assessment  battery  upon  return.  Cadet 
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supervisors  and  peers  also  assessed  cadet  perfonnance  in  a  rate/rank  questionnaire  as  part  of  the 
post-test. 

Findings: 

Five  unique,  but  related,  factors  of  cross-cultural  competence  were  identified  through  the 
research.  The  resultant  model  serves  as  the  foundation  for  an  assessment  methodology  that 
includes  self-report  measures,  situational  judgment  tests,  and  cultural  vignettes.  The  model  and 
metrics  provided  the  foundation  for  the  creation  of  a  computer-based  version  of  the  assessment 
battery,  or  Cross-Cultural  Assessment  Tool  (C-CAT).  This  online  program  assesses  an 
individual  Soldier’s  cross-cultural  competence  and  provides  mission-relevant  competence 
ratings,  descriptions  of  individual  strengths/weaknesses  and  how  these  can  impact  mission 
perfonnance,  and  suggestions  for  improvement. 

Utilization  and  Dissemination  of  Findings: 

The  current  study  led  to  the  development  of  a  refined  3C  model  that  characterizes  the 
main  components  of  3C  and  defines  the  varying  levels  of  cross-cultural  competence.  This  model 
may  inform  future  research  efforts  related  to  assessment  development,  wherein  it  is  critical  to 
understand  and  articulate  the  main  constructs  that  comprise  3C.  Additionally,  the  current 
research  resulted  in  the  creation  of  the  C-CAT  which  is  one  of  the  few  military-specific  cross- 
cultural  assessment  batteries  available  to  date.  The  use  of  this  particular  battery  may  assist  in  the 
identification  of  gaps  in  individual  cross-cultural  competence  by  highlighting  areas  of 
opportunity  for  improvement  and  suggestions  for  how  to  improve.  Given  that  the  customized 
feedback  provided  by  the  C-CAT  is  based  on  an  individual’s  performance  on  the  battery,  results 
are  directly  applicable  to  the  user.  Finally,  the  C-CAT  can  be  used  longitudinally  to  assess 
changes  in  cross-cultural  competence  over  time. 
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MEASURING  LEARNING  AND  DEVELOPMENT  IN  CROSS-CULTURAL  COMPETENCE 


For  Soldiers  in  the  contemporary  operational  environment,  perfonnance  beyond  typical 
military  technical  and  tactical  skills  is  increasingly  required.  For  example,  Soldiers  must  interact 
with,  engage,  and  persuade  local  populations  in  foreign  countries  to  achieve  United  States  Anny 
small  unit  leader  mission  objectives.  The  Anny  leadership  recognizes  this  and  is  increasingly 
prioritizing  cross-cultural  training  for  Soldiers.  While  many  cultural  training  initiatives  are 
currently  operational,  the  lack  of  a  sound  method  for  assessing  cross-cultural  competence  (3C)  in 
Soldiers  leaves  their  effectiveness  uncertain.  To  ensure  that  Soldiers  are  adequately  prepared  for 
the  challenges  of  active  duty  in  unfamiliar  cultures,  3C  must  be  clearly  defined  and  assessed  at 
the  individual  level.  With  these  objectives  in  mind,  this  Small  Business  Innovation  Research 
(SBIR)  research  effort  had  two  main  thrusts:  (1)  model  the  development  and  components  of  3C 
at  the  tactical  level,1  and  (2)  develop  a  battery  of  metrics  that  reliably  and  validly  assesses 
Soldiers’  3C. 

The  research  community  has  struggled  to  agree  on  a  consistent  definition  of  3C  (see 
Abbe,  Gulick,  &  Hennan  (2007)  for  a  discussion  of  this  issue).  To  accurately  develop  a  model 
of  3C  and  build  a  metric  that  can  assess  it,  an  operational  definition  of  3C  that  captures  the  field 
requirements  of  deployed  military  personnel  in  novel  cultural  environments  must  be  identified. 

To  this  end,  the  research  team  employed  a  rigorous  mixed  method  approach  to  identify  the 
components  of  operational  3C  associated  with  effective  perfonnance  in  the  field  (Phase  I  of  this 
(SBIR)  effort).  Critical  incidents  were  elicited  wherein  Soldiers  described  the  challenging 
aspects  of  cross-cultural  encounters  or  assessments.  In  addition,  team  ranking  task  interviews 
were  conducted  during  which  Soldiers  evaluated  their  peers  based  on  perceived  cultural  ability, 
as  well  as  simulation  interviews  during  which  Soldiers  responded  to  cognitively  authentic 
deployment  scenarios.  These  data  were  compiled  and  analyzed  with  a  focus  on  identifying 
patterns  and  frequencies  in  the  presence  and/or  mention  of  specific  KSAAs  (knowledge,  skills, 
abilities,  and  aptitudes).  This  yielded  an  initial  set  of  29  cognitive,  affective,  and  behavioral 
KSAAs,  which  were  later  comprised  to  fonn  an  overall  3C  construct.  Within  the  preliminary  3C 
framework  itself,  four  levels  of  competence  were  represented.  Those  levels,  in  order  of 
increasing  competence,  were  precompetent,  intermediate,  proficient,  and  advanced.  Each  level 
of  competence  was  described  in  terms  of  the  KSAAs  that  were  present  (or  absent)  in  each 
component,  as  well  as  the  levels  of  development  for  KSAAs  within  each  component.  A  detailed 
description  of  the  interview  collection  techniques,  the  initial  set  of  29  KSAAs,  and  the  four 
levels  of  3C  development  can  be  found  in  McCloskey,  Grandjean,  Behymer,  and  Ross  (2009). 

The  initial  focus  of  Phase  II  was  to  refine  the  Phase  I  model  based  on  the  data  collected  in 
Phase  I.  First,  the  initial  set  of  29  KSAAs  was  reduced  to  16  KSAAs.  Researchers  compared  the 
list  of  29  KSAAs  to  40  cultural  learning  statements  and  their  definitions  that  were  derived  by  a 


1  Although  cross-cultural  competence  is  important  at  the  operational  and  strategic  levels  and  future  research  should 
examine  its  impact  on  performance,  tactical  operations  requiring  direct  contact  and  interaction  with  local  populaces 
is  an  ideal  starting  point  for  describing  cross-cultural  competence.  Small  unit  leaders  are  often  the  most  prominent 
representatives  of  American  foreign  policy  and  their  decisions  influence  not  only  immediate  tactical  situations  but 
higher  operational  and  strategic  levels  (Krulak,  1999).  For  a  more  in-depth  discussion  see  McCloskey,  Behymer, 
Papautsky,  Ross,  &  Abbe  (2010). 
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Department  of  Defense  working  group  of  researchers  and  operators  (McDonald,  McGuire, 
Johnston,  Selmeski,  &  Abbe,  2008).  From  this  review,  we  consolidated  and  revised  the  list  to  a 
smaller  set  of  16  KSAAs  that  we  believed  had  less  overlap,  were  relevant  to  the  cross-cultural 
competence,  and  would  be  useful  for  further  research. 

Based  on  feedback  from  potential  end  users  who  suggested  that  a  16-factor  model  was 
too  complex  for  their  needs  and  that  a  more  parsimonious  model  was  needed,  we  continued  to 
examine  how  to  consolidate  this  group  of  16  factors.  Our  next  step  was  to  generate  a  self-report 
measure;  each  of  the  16  KSAAs  (and  a  social  desirability  scale)  were  measured  with  eight  items 
(four  general  items  and  four  deployment-oriented  items)  for  a  total  of  136  items  based  on  Clark 
and  Watson’s  (1995)  suggestion  that  an  item  pool  must  be  broad  enough  to  cover  the  targeted 
content  area.  Four  of  these  questions  prompted  participants  to  report  on  current  behaviors  and 
preferences  (general  items),  whereas  the  other  four  questions  asked  participants  about 
hypothetical  or  future  situations  (deployment-oriented  items).  To  develop  items  for  this 
measure,  we  examined  items  that  sorters  agreed  addressed  a  specific  KSAA  from  the  team 
ranking  interview,  existing  items  in  the  literature,  and  raw  data  from  the  critical  incident 
interviews.  Items  addressed  agreement  on  what  each  cadet  would  do  in  a  deployment  situation 
and  other  potential  behaviors  on  a  6-point  Likert  scale  (1  =  Strongly  Disagree ;  6  =  Strongly 
Agree).  The  Willingness  to  Engage  subscale  is  shown  below  as  an  example.  The  first  4 
questions  are  deployment-oriented,  while  the  next  four  are  general. 

Willingness  to  Engage: 

1 .  Prior  to  a  deployment,  I  would  try  to  learn  the  basics  of  the  language  before  going, 
whether  directed  to  or  not. 

2.  During  deployments,  I  would  seek  out  opportunities  to  experience  the  local  culture. 

3.  If  deployed,  I  would  enjoy  opportunities  to  interact  with  the  people  of  that  country 
and  leam  about  their  lives. 

4.  If  deployed,  I  would  avoid  eating  and  socializing  with  the  locals. 

5.  In  general,  I  try  to  limit  my  interactions  with  strangers. 

6.  I  enjoy  meeting  people  who  are  different  from  me. 

7.  When  I  am  meeting  new  people,  I  am  willing  to  try  new  activities  I  wouldn’t 
otherwise  try. 

8.  I  tend  to  get  very  engaged  when  part  of  a  group  discussion. 

Due  to  time  and  access  to  participant  limitations,  our  sample  size  for  this  initial  analysis 
was  43  Soldiers,  which  was  not  large  enough  for  a  factor  analysis.  However,  it  did  allow  us  to 
examine  the  correlations  between  the  constructs.  This  examination  resulted  in  a  reduction  of  the 
KSAAs  to  five  main  factors:  Cultural  Maturity,  Cognitive  Flexibility,  Cultural  Knowledge, 
Cultural  Acuity,  and  Interpersonal  Skills.  Figure  1  shows  the  KSAA  consolidation  process. 
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Figure  1.  3C  KSAA  consolidation. 


Additionally,  the  3C  developmental  model  (which  detailed  the  development  of  KSAAs  at 
each  of  four  competency  levels)  was  modified  based  on  both  reviews  of  the  expertise 
development  literature  and  the  collected  data.  The  resulting  model  is  shown  in  Figure  2.  This 
figure  depicts  differing  levels  of  3C  as  concentric  gradated  pentagons,  one  for  each  level.  For 
example,  the  inner  area  with  the  lightest  shading  represents  a  Level  1,  Precompetent  Soldier.  An 
individual  at  this  level  may  not  be  ready  to  benefit  from  the  cultural  knowledge  and  basic 
guidelines  upon  which  more  advanced  Soldiers  depend  due  to  attitudes  that  impede  the 
acquisition  of  this  knowledge  (i.e.,  learning).  Only  if  individuals  are  past  the  Precompetent  level 
can  they  begin  to  learn  the  important  knowledge  and  skills  necessary  to  achieve  higher  levels  of 
competency.  Moving  outward,  increases  in  shading  density  represent  higher  levels  of  overall 
3C.  Thus,  Level  2  is  more  competent  than  Level  1,  and  so  forth.  Increases  in  each  of  the  five 
factors,  however,  occurs  at  different  rates  for  different  levels.  For  example,  the  most  substantial 
difference  distinguishing  a  Level  1  from  Level  2  Soldier  is  Cultural  Maturity.  Distinguishing 
Level  3  from  Level  2,  all  factors  show  improvement,  with  the  most  substantial  increase  being  in 
Interpersonal  Skills.  The  overt  behavioral  abilities  comprising  Interpersonal  Skills  will  be  well- 
developed  for  the  Level  3  Soldier,  while  factors  such  as  Cultural  Acuity  and  Cognitive 
Flexibility,  which  are  more  cognitively  based,  will  still  be  limited.  Finally,  it  is  in  Level  4 
(Advanced),  the  darkest-shaded  outer  area,  where  cognitive  factors  (Cultural  Knowledge, 
Cognitive  Flexibility,  and  Cultural  Acuity)  show  the  greatest  improvement,  and  it  is  at  this  level 


3 


that  a  Soldier  is  most  accurate  in  his  or  her  cross-cultural  assessments  and  the  most  effective  in 
actions.  Cultural  Maturity  differences  between  a  Level  3  and  a  Level  4  Soldier  will  be  smaller, 
as  the  Level  3  Soldier  will  have  already  developed  high  emotional  self-regulation  and 
willingness  to  engage.  A  complete  description  of  this  process  and  the  resulting  model  can  be 
found  in  McCloskey,  Behymer,  Papautsky,  Ross,  and  Abbe  (2010). 


Figure  2.  Initial  developmental  model  of  3C. 


Although  identification  of  the  aforementioned  framework  was  essential  to  the  conceptual 
development  of  3C,  there  still  exists  a  lack  of  means  by  which  to  assess  it.  Thus,  the  purpose  of 
the  present  study  (in  fulfillment  of  Phase  II)  was  to  develop  a  measure  of  3C,  using  the 
previously  proposed  framework  as  a  theoretical  guide.  Multiple  data  collections  with  Soldiers 
and  ROTC  cadets  were  conducted  to  refine  and  validate  the  measures.  These  data  collections  led 
to  a  final  refinement  of  the  components  of  3C  and  the  levels  of  development  within  the  model. 
Additionally,  the  cadet  data  established  that  participants’  responses  were  consistent  with  peer 
and  supervisor  perceptions  of  their  actual  performance  in  the  field,  thus  providing  evidence  of 
criterion-related  validity.  Findings  from  the  Soldier  data  allowed  the  research  team  to  establish 
construct  validity  as  the  different  measures  of  the  battery  were  significantly  correlated.  Finally,  a 
computer-based  version  of  the  assessment  battery  was  created  that  included  the  development  of 
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customized  feedback  based  on  an  individual’s  performance  on  the  battery.  This  report  describes 
the  measures  used  in  the  data  collections,  the  processes  and  findings  underlying  their 
modification,  and  the  resulting  model  of  3C,  as  well  as  the  computer-based  assessment  battery 
that  can  be  utilized  to  assess  3C  in  military  personnel  preparing  for  deployment. 

Method 


Participants 

Approximately  270  Soldiers  (missing  data  points  account  for  the  variable  sample  sizes 
throughout  this  report),  the  majority  of  whom  were  male  (97%)  ranging  in  age  from  19  to  52  (M 
=  30.59,  SD  =  5.72),  were  given  our  assessment  battery.  This  sample  was  selected  based  on 
several  factors.  As  stated  in  the  introduction,  our  target  respondents  were  small  unit  leaders 
operating  at  the  tactical  level.  Thus,  Captains  and  Non-Commissioned  Officers  comprised  83% 
of  the  data,  with  the  most  common  grade  being  Captain  (36%),  Staff  Sergeant  (19%),  and 
Sergeant  First  Class  (19%).  Additionally,  we  wanted  to  ensure  that  a  large  variety  of  Military 
Occupational  Specialties  were  represented  (a  complete  list  of  the  MOSs  of  the  Soldiers 
interviewed  can  be  found  in  Appendix  C,  with  Annor  (26%)  and  Civil  Affairs  (18%)  being  the 
most  common).  Finally,  due  to  the  nature  of  Soldier  availability,  convenience  played  a  role  in 
the  selection  of  participants.  The  majority  of  the  Soldiers  were  Caucasian  (80%),  however,  other 
racial/ethnic  groups  were  also  represented,  including  African  Americans  (10%),  Latinos  (5%), 
Asians  (3%),  Native  Americans  (2%),  and  Pacific  Islanders  (<  1%).  Fifty-three  percent  of 
Soldiers  reported  their  ASVAB  general  technical  score,  with  an  average  of  1 14.22  (SD  =  1 1.56). 
In  general,  Soldiers  averaged  eight  years  of  service  (SD  =  5.22)  and  more  than  one  deployment. 
Exposure  to  cultural  training  was  quite  varied  across  Soldiers.  More  than  three-quarters  (77%) 
reported  having  received  more  than  1  hour  of  cultural  training;  however,  fewer  than  half  of  the 
Soldiers  (41%)  reported  more  than  1  hour  of  foreign  language  training.  The  amount  of  cultural 
training  ranged  from  0  to  200  hours.  The  amount  of  foreign  language  training  received  ranged 
from  0  to  1,000  hours. 

Our  sample  also  included  an  additional  171  ROTC  cadets.  These  cadets  participated  in  a 
2-week  cultural  immersion  program  where  they  interacted  with  local  populaces  on  a  variety  of 
humanitarian  missions  (locations  included  China,  Ghana,  Russia,  Senegal,  Mongolia,  Egypt, 
Morocco,  Taiwan,  and  Tanzania).  At  the  request  of  Cadet  Command,  we  also  attempted  to  use 
the  battery  to  assess  the  cadets’  levels  of  3C  both  pre-deployment  and  post-deployment  to  gauge 
the  effectiveness  of  the  cultural  immersion  program.  According  to  Cadet  Command,  at  the  time 
of  their  deployment,  the  s,  regardless  of  their  scholarship  status,  had  received  the  same  amount  of 
cultural  training.  This  opportunity  provided  us  with  the  ability  to  examine  the  reliability  of  our 
assessment  battery,  as  well  as  the  chance  to  establish  construct  validity,  as  we  were  able  to 
collect  both  peer  and  supervisor  ratings  for  the  cadets.  Because  the  effectiveness  of  the  cultural 
immersion  program  was  not  directly  relevant  to  the  development  of  the  3C  model  or  the 
assessment  battery,  the  results  detailing  the  impact  of  the  cultural  immersion  program  on  the 
cadets’  performance  on  our  battery  is  provided  in  Appendix  D. 

Cadets  were,  on  average,  20  years  old  (+/-  1  year).  Only  15%  of  cadets  reported  their 
ASVAB  general  technical  score,  with  an  average  of  96.04.  Although  the  majority  of  the  cadets 
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were  Caucasian  (87%),  other  racial/ethnic  groups  were  also  represented  including  Asian  (5%), 
African  American  (5%),  Hispanic  (2%),  Native  American  (1%),  and  Pacific  Islander  (1%).  Most 
cadets  (57%)  reported  two  previous  trips  abroad  and  had  a  moderate  level  of  prior  cultural 
interaction  (M  =2.88,  SD  =  1.11  on  a  scale  of  1  to  5  where  1  =  No  Interaction  and  5  =  Almost 
Constant  Interaction).  Approximately  one-third  of  the  sample  (34%)  reported  knowing  no 
languages  other  than  English. 

Throughout  the  Results  section  of  the  report  we  will  indicate  which  results  are  based 
solely  on  cadet  data  and  which  are  based  on  a  combination  of  Soldier  and  cadet  data.  For  some 
analyses,  the  two  data  sets  were  combined  because  the  advantages  (increased  statistical  power, 
increasing  the  range  of  possible  skill  level)  outweighed  the  disadvantages  (the  difference 
between  Soldiers  and  cadets).  Additionally,  the  fact  that  3C  does  not  necessarily  increase  as  a 
result  of  deployment  experience,  amount  of  cross-cultural  interaction,  or  years  of  service  served 
to  minimize  the  differences  between  Soldiers  and  cadets  in  tenns  of  3C  ability. 

Materials 

The  measures  used  in  the  current  effort  were  designed  to  be  comprehensive  in  gathering 
information  on  participants’  (both  Soldiers  and  cadets)  demographics,  backgrounds,  and 
experiences,  as  well  as  actions  taken  in  potential  deployment  situations  collected  via  both 
quantitative  self-report  measures  and  qualitative  scenario-based  measures  of  situational  judgment 
tests  (SJTs)  and  vignettes.  Described  below  are  the  measures  of  the  3C  assessment  battery  that 
were  administered  to  Soldiers  and  cadets.  The  implemented  modifications  and  the 
configurations  of  measures  (versions  of  the  assessment  battery)  that  were  administered  to  the 
different  samples  of  participants  are  described  in  subsequent  sections. 

Demographics  questionnaire.  A  demographics  questionnaire  developed  as  part  of  the 
Phase  I  effort  was  included  in  the  assessment  battery.  Participants  were  asked  to  report  their  age, 
gender,  ASVAB  general  technical  score,  and  race/ethnicity.  In  addition,  participants  were  asked 
to  describe  the  amount  of  cultural  interaction  they  had  had,  time  lived  outside  of  the  United 
States,  amount  of  international  travel  conducted  over  their  lifetimes,  and  proficiency  in  a 
language(s)  other  than  English  using  a  5-item  Likert  scale  ranging  from  1 :  Novice  -  know  a  few 
words  and/or  phrases  to  5:  Native  Speaker  -  can  speak  and  understand  the  language  as  a  native. 
An  open-ended  item  was  also  included  that  allowed  participants  to  report  any  additional  cross- 
cultural  experiences,  such  as  having  been  bom  abroad,  having  lived  in  a  culturally  diverse 
environment,  or  having  parents  who  are/were  foreign  nationals  or  naturalized  citizens. 

Self  report  questionnaire.  A  Likert-scale  self-report  questionnaire  was  developed  to 
assess  3C.  The  content  was  infonned  by  critical  incident  and  team  ranking  data  from  Soldiers  in 
Phase  I,  as  well  as  a  review  of  the  literature  also  conducted  during  Phase  I.  Soldiers’  responses 
that  reflected  specific  KSAAs  as  determined  by  the  research  team  (McCloskey  et  ah,  2010)  were 
adapted  into  self-report  items.  For  example,  a  Soldier  described  a  competent  team  member  as 
“seek(ing)  out  opportunities  to  experience  the  local  culture.”  This  statement  was  then  converted 
into  a  self-report  item:  “During  deployments,  I  would  seek  out  opportunities  to  experience  the 
local  culture.” 
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The  measure  initially  consisted  of  sixteen  subscales  (one  for  each  of  the  previously 
identified  KSAAs)  with  eight  items  per  subscale.  The  instrument  was  pilot-tested  and  items  that 
reduced  the  reliability  of  a  subscale  were  eliminated.  The  resulting  instrument  included  79  items 
assessing  participants’  3C.  Items  addressed  participant  agreement  with  statements  describing 
what  he  or  she  would  do  in  a  deployment  or  other  related  situation  on  a  6-point  Likert  scale  ( 1  = 
Strongly  Disagree',  6  =  Strongly  Agree).  Based  on  theory  and  empirical  data  collected  from 
Soldiers  in  Phase  I,  items  were  categorized  into  five  hypothesized  components  of  cross-cultural 
competence  of  Cultural  Maturity  (thirteen  items),  Cognitive  Flexibility  (eight  items),  Cultural 
Knowledge  (six  items),  Cultural  Acuity  (sixteen  items),  and  Interpersonal  Skills  (twenty  items). 
In  addition,  three  items  pertained  to  Cultural  Efficacy  (Abbe,  Geller,  and  Everett,  2010),  five 
items  pertained  to  Learning  Orientation  and  eight  items  comprised  a  social  desirability  scale 
(Crowne  &  Marlowe,  1960).  The  social  desirability  scale  was  included  only  to  identify  and 
eliminate  participants  who  appeared  to  not  be  answering  truthfully  (e.g.,  responding  “Strongly 
Agree”  to  questions  like  “I’m  always  willing  to  admit  it  when  I  make  a  mistake”). 

Situational  Judgment  Tests .  To  address  the  limitations  of  a  self-report  only  assessment 
approach  (e.g.,  social-desirability  bias,  limited  to  measuring  self-perceptions),  three  SJTs  were 
developed  following  a  framework  proposed  by  Motowidlo,  Dunnette,  and  Carter  (1990).  SJTs 
have  the  ability  to  assess  job  relevant  behaviors  and  have  been  shown  to  be  one  of  the  best 
predictors  of  job  performance  (McDaniel  &  Nguyen,  2001).  A  complete  description  of  our  SJT 
developmental  process  can  be  found  in  McCloskey  and  Behymer  (2010). 

Content  validity  of  the  SJTs  was  established  by  using  situations  that  were  based  on  actual 
critical  incidents  gathered  from  Soldiers  in  cognitive  task  analysis  interviews.  The  situations 
were  adapted  to  a  concise  story-like  fonnat  that  included  multiple  decision  points  (e.g.,  in  a 
situation  where  a  prized  village  goat  is  struck  by  a  US  vehicle:  As  the  squad  leader,  what 
immediate  courses  of  action  (COAs)  would  you  take  in  this  situation  and  why?).  Eighty  Soldiers 
were  then  asked  to  provide  open-ended  responses  to  the  decision  points  in  order  to  support  the 
development  of  realistic  multiple-choice  options.  These  responses  were  then  consolidated  into 
five  to  eight  representative  options.  Next,  a  separate  sample  of  56  Soldiers  ranked  each  possible 
response  on  both  tactical  and  cultural  effectiveness.  This  allowed  the  research  team  to  retain 
options  that  were  equally  effective  from  a  tactical  standpoint  but  varied  significantly  in  cultural 
effectiveness.  For  example,  one  common  response  to  the  goat-strike  query  was  along  the 
following  lines:  Through  the  interpreter,  I  would  apologize  to  the  group  for  hitting  the  goat  and 
have  the  interpreter  offer  to  pay  for  the  animal.  Such  a  response  reflects  a  moderate  degree  of 
cultural  competence.  Conversely,  a  response  that  represents  lower  cultural  competence  was:  / 
would  have  my  men  get  back  in  the  trucks  and  leave  to  go  straight  to  the  village  and  secure  it 
because  our  mission  is  to  find  that  weapons  cache,  so  we  could  deal  with  the  goat  problem  later. 
In  the  finalized  measure,  participants  were  asked  to  identify  what  their  Most  Likely  and  the  Least 
Likely  courses  of  action  would  be  from  several  multiple  choice  options  at  key  decision  points 
within  a  scenario.  The  scoring  system  is  shown  in  Table  1. 


Table  1 

Scoring  System  for  Situational  Judgment  Tests 
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Participant’s 

response 

Best  course  of  action 

Worst  course  of 
action 

Other  course  of  action 

Most  likely 

3 

1 

2 

Least  likely 

1 

3 

2 

Thus,  if  a  participant  selected  the  “best”  course  of  action  as  the  action  they  would  most 
likely  take,  they  would  receive  three  points.  If  they  selected  the  “worst”  course  of  action  as  the 
action  they  would  most  likely  take,  they  would  receive  one  point.  Any  other  response  was  given 
two  points.  The  total  score  for  each  question  was  the  sum  of  the  points  the  participant  received 
for  their  most  likely  and  least  likely  response.  The  overall  score  of  the  SJT  was  calculated  by 
averaging  the  scores  across  all  questions. 

Three  SJTs  were  developed  and  utilized  in  the  Phase  II  effort.  In  The  Goat  Strike  (SJT 
1),  a  situation  was  presented  where  Soldiers  in  HUMVEEs  struck  and  killed  a  goat  (three 
decision  points).  Looking  for  Weapons  (SJT  2)  presented  a  situation  where  a  Soldier  and  his/her 
platoon  are  assigned  to  visit  a  village  that  is  rumored  to  house  a  major  weapons  cache  (five 
decision  points).  SJT  3,  Restructuring,  presented  a  scenario  in  which  a  Soldier  and  his/her 
platoon  are  assigned  to  a  country  that  is  in  the  process  of  restructuring  in  order  to  train  the 
national  police  (five  decision  points). 

Cultural  Vignettes.  In  addition  to  SJTs,  which  are,  by  nature,  multidimensional,  the 
research  team  sought  to  develop  a  scenario-based  measure  that  could  potentially  be  tailored  to 
focus  on  specific  factors.  The  intent  was  to  construct  a  measure  that  would  capitalize  on  the 
advantages  of  both  SJTs  (scenario-based)  and  self-report  (individual  questions  onto  which 
specific  factors  can  load).  To  this  end,  the  research  team  developed  several  cultural  vignettes 
that  are  similar  in  content  to  SJTs,  but  were  specifically  based  on  the  five  factors  in  the  3C 
developmental  model  (i.e.,  Cultural  Maturity,  Cognitive  Flexibility,  Interpersonal  Skills,  Cultural 
Knowledge,  and  Cultural  Acuity).  Whereas  the  SJT  scenarios  and  response  options  were  based 
on  critical  incidents  and  overall  effectiveness  ratings  provided  by  Soldiers,  the  vignettes  were 
developed  by  the  research  team.  The  content  and  response  options  were  still  based  on  data 
gathered  from  Soldiers,  but  the  focus  from  the  start  was  on  tailoring  situation  and  question 
specifics  to  target  individual  factors.  The  cultural  vignettes  also  differed  from  SJTs  in  that 
participants  were  asked  to  rank  a  range  of  four  response  options  from  1  (Most  Likely  Response) 
to  4  (Least  Likely  Response)  for  each  vignette  question.  The  four  response  options  were 
obtained  directly  from  Soldiers’  responses  to  simulation  interview  questions  that  had  been 
gathered  and  rated  by  the  research  team.  Each  response  option  represented  one  of  the  four  levels 
of  cultural  competence  (Precompetent,  Intennediate,  Proficient,  Advanced).  The  vignette 
scoring  system  is  given  in  Table  2. 


Table  2 

Scoring  System  for  Vignette  1,  Question  1 


2  While  the  research  team  attempted  to  create  factor-targeted  vignettes,  we  recognize  that  they  still  may  retain  multi¬ 
dimensionality  issues  similar  to  SJTs.  This  is  a  potential  topic  for  further  research. 


8 


Vignette  Question  Option 

Participant’s 

response 

Answer 

key 

Absolute 

difference 

Sum  of 

absolute 

differences 

Reversed 

score 

1  1  A 

2 

2 

0 

1  1  B 

3 

4 

1 

c 

1  1  C 

1 

1 

0 

z 

o 

1  1  D 

4 

3 

1 

For  the  question  represented  in  Table  2,  Option  C  is  the  best  answer,  followed  by  Option 
A,  then  Option  D;  Option  B  is  the  worst  answer.  The  participant  correctly  selected  Option  C  as 
the  best  response  and  Option  A  as  the  second  best.  However,  the  participant  selected  the  worst 
option  (Option  B)  over  Option  D.  For  each  response  the  absolute  value  of  the  difference 
between  the  participant’s  answer  and  the  correct  answer  is  calculated  and  summed.  In  this  case, 
the  participant’s  score  was  2  (the  sum  of  the  values  in  the  Absolute  Difference  column, 

0+1+0+1).  If  the  participant  had  correctly  identified  the  order  (C,  A,  D,  B),  his  or  her  score 
would  have  been  0.  If  the  participant  had  answered  the  worst  way  (B,  D,  A,  C),  his  or  her  score 
would  have  been  8.  To  avoid  the  confusion  associated  with  having  a  scoring  system  in  which 
lower  values  are  better,  we  reverse  scored  the  participant’s  score.  Thus,  if  a  participant  scored  a 
0,  the  reverse  score  would  be  an  8,  if  they  scored  2,  the  reverse  would  be  a  6,  and  so  on.  An 
overall  vignette  score  is  calculated  by  averaging  the  participant’s  score  on  each  question,  with 
higher  scores  indicative  of  higher  levels  of  competence. 

Five  vignettes  were  used  in  the  Phase  II  effort:  The  New  Interpreter  (Vignette  1), 
Problems  With  Training  (Vignette  2),  Meeting  With  Leaders  (Vignette  3),  Burn  Victim  Support 
(Vignette  4),  and  Needs  Assessment  Gone  Wrong  (Vignette  5).  The  New  Interpreter  vignette 
presented  a  scenario  where,  upon  Soldiers’  arrival  in  Honduras,  the  interpreter  exhibits  several 
unexpected  behaviors  (five  decision  points).  Problems  With  Training  presented  a  scenario  in 
which  Soldiers  faced  challenges  associated  with  designing  and  managing  a  training  program 
(four  decision  points).  Meeting  With  Leaders  presented  a  scenario  where  Soldiers  enter  a  village 
to  meet  with  its  leaders  and  are  faced  with  unexpected  challenges  (four  decision  points).  Burn 
Victim  Support  presented  a  scenario  in  which  Soldiers  are  assigned  to  provide  medical  support  to 
burn  victims  (five  decision  points).  Needs  Assessment  Gone  Wrong  is  a  scenario  in  which  a 
team,  led  by  the  respondent,  is  sent  on  a  humanitarian  mission  to  distribute  food  and  supplies  to  a 
series  of  remote  villages.  . 


Peer  and  Supervisor  Rating/Ranking  Scale.  To  be  considered  a  valid  measure  of  3C, 
our  assessment  battery  needed  to  be  compared  to  a  measure  of  performance,  in  this  case, 
performance  evaluations  by  Soldiers’  superiors  and  their  peers  on  culturally- immersive  tasks. 
Due  to  unanticipated  changes  in  deployment  schedules,  however,  we  were  unable  to  obtain  peer 
and  supervisors  ratings  from  the  Soldiers  in  our  sample  during  the  period  of  performance. 
Continuing  research  (described  below)  is  planned  with  an  additional  sample  of  Civil  Affairs 
Soldiers  to  examine  the  extent  to  which  the  assessment  battery  correlates  with  peer  and 
supervisor  ratings. 


9 


Fortunately,  we  were  able  to  obtain  peer  and  supervisor  evaluations  from  the  cadets  in 
our  sample.  The  peer/supervisor  measure  is  provided  in  Appendix  B.  To  collect  peer  and 
supervisor  evaluations,  a  behaviorally  anchored  rating  scale  (BARS)  was  constructed  for  six 
training  objectives  in  a  cross-cultural  context.  The  objectives  included  culture  fundamentals, 
culture  awareness,  culture  skills,  communication  skills,  cross-cultural  operations,  and  influence. 
Because  this  peer  and  supervisor  measure  was  used  only  with  cadets,  the  source  of  the  focus 
areas  was  the  Army’s  ROTC  Culture  and  Language  Training  Objectives,  which  are  based  on  the 
Anny  Culture  and  Foreign  Language  Strategy  (ACFLS).  For  each  training  objective,  four 
behavioral  anchors  were  developed  corresponding  to  four  levels  of  competence.  Examples  for 
the  behavioral  anchors  were  drawn  from  multiple  previously  collected  data  sources  (critical 
incidents,  team  rankings,  etc.).  Below  is  an  excerpt  from  the  instructions  that  respondents  were 
given: 


Please  provide  a  rating  for  yourself  and  each  squad-mate  by  placing  his/her  name  in  a 
category  corresponding  to  a  description  that  best  represents  his/her  level  of  performance. 
Multiple  individuals  may  belong  to  the  same  category.  Please  try  to  use  all  categories  to 
span  the  continuum  of  performance.  In  addition,  within  each  category,  rank  the 
individuals  based  on  their  level  of  effectiveness  relative  to  each  other. 

The  instructions  were  followed  by  a  detailed  example.  The  respondents  were  then 
provided  with  an  empty  rating  table  for  each  question  within  which  they  were  asked  to 
simultaneously  rate  and  rank  their  peers. 

Two  versions  of  the  rating/ranking  scale  were  developed.  One  version  had  four 
categories  (4-point  scale)  and  another  version  had  eight  categories  (8-point  scale)  -  two  per  each 
behavioral  anchor.  Both  versions  were  piloted,  and  it  was  detennined  that  the  4-point  scale  was 
sufficient  to  provide  enough  variability  in  responses.  The  pilot  test  was  conducted  with  19 
cadets  at  a  local  Anny  ROTC  program.  All  participants  were  juniors  or  seniors  at  Wright  State 
University  in  Dayton,  OH. 

The  purpose  of  the  pilot  test  was  to  assess  elements  of  the  cross-cultural  assessment 
battery.  Specifically,  we  were  interested  in  exploring  the  clarity  of  the  instructions  for  a  peer¬ 
ranking/rating  measure,  the  relative  difficulty,  and  the  time-to-completion  of  the  assessment 
battery.  The  rate/rank  questionnaire  modifications  were  informed  by  response  errors  and 
participant  feedback. 

Procedure 

A  member  of  the  research  team  briefed  the  participants  on  the  purpose  of  the  study. 
Participants  were  provided  with  a  Privacy  Act  Statement  and  were  asked  to  sign  a  consent  form. 
The  researcher  then  administered  the  assessment  battery.  Participants  filled  out  the  battery  in  a 
quiet  location  over  a  period  of  approximately  45  minutes  to  1  hour. 

For  both  Soldiers  and  cadets,  materials  included  the  3C  assessment  battery  comprised  of 
a  demographics  questionnaire,  self-report,  SJTs,  and  vignettes  (as  described  in  the  Materials 
section  of  this  report).  However,  several  modifications  were  made  to  the  items  within  the  test 
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battery  administered  to  cadets  to  better  relate  to  the  cadets’  experiences  (e.g.,  modifying  some 
questions  to  address  humanitarian,  rather  than  tactical,  missions).  Pre-deployment  and  post¬ 
deployment  versions  of  the  3C  assessment  battery  were  developed.  Both  the  pre-deployment  and 
post-deployment  versions  included  a  self-report  questionnaire. 

Vignettes  4  (Bum  Victim  Support)  and  5  (Needs  Assessment  Gone  Wrong)  were 
developed  specifically  for  purposes  pertaining  to  the  cadet  data  collection  following  the  same 
development  procedure  described  above.  The  SJTs  and  vignettes  were  balanced  for  difficulty 
across  pre-test  and  post-test  by  examining  means  and  standard  deviations.  For  example,  the 
average  score  on  SJT  2  was  not  significantly  different  than  the  average  score  on  SJT  3, 
suggesting  that  they  had  a  similar  level  of  difficulty. 

Cadets  completed  a  pre-test  version  of  a  cross-cultural  competence  assessment  battery, 
traveled  abroad  for  approximately  2  weeks,  and  immediately  completed  a  post-test  version  of  the 
assessment  battery  upon  return.  Cadet  supervisors  and  peers  also  completed  the  supervisor 
rate/rank  questionnaires  as  part  of  the  post-test. 

Results 


Model  Refinement 

To  achieve  a  comprehensive  and  reliable  shortened  version  of  the  self-report  scale  and  to 
test  the  3C  model,  the  research  team  conducted  a  series  of  exploratory  factor  analyses  on  a 
combined  data  set  of  Soldier  and  pre-test  cadet  data.  The  increased  sample  size  allowed  for  a 
thorough  examination  of  the  assessment  items.  While  we  were  aware  that  these  are  two  different 
samples,  we  also  recognized  that  several  of  the  Soldiers  in  our  sample  were  roughly  the  same  age 
as  our  cadets,  and  while  the  cadets  had  not  yet  experienced  actual  deployments,  some  of  the 
participants  in  the  Soldier  sample  also  had  not  yet  been  deployed.  We  considered  the  differences 
in  samples  and  decided  that  the  value  in  increasing  sample  size  warranted  combining  samples  at 
this  stage  in  the  research.  Factor  analyses  allowed  the  team  to  conclude  that  the  resultant  item 
structure  was  consistent  with  the  five  hypothesized  components.  The  final  measure,  the 
refinement  of  which  is  described  below,  is  comprised  of  44  items  and  conformed  to  a  5-factor 
structure. 

An  exploratory  approach  to  scale  construction  was  taken  (Tellegen  &  Waller,  2008)  by 
allowing  both  theory  and  data  to  inform  the  final  self-report  measure.  Data  collected  from 
Soldiers  and  cadets  in  Phase  II  were  utilized  to  increase  internal  consistency  by  item  elimination. 
As  the  goal  was  to  create  a  multi-scale  battery,  the  team  used  the  data  to  empirically  enhance  the 
independence  and  distinctiveness  of  the  scales  by  conducting  a  series  of  exploratory  factor 
analyses. 

The  researcher  hypothesized  five  factors  that  loaded  onto  the  items  as  previously 
described.  The  hypotheses  were  drawn  from  existing  literature  and  from  extensive  interviews 
with  Soldiers  with  deployment  experiences.  These  hypothesized  scales  remained  intact 
throughout  Phase  II  as  scale  reliabilities  and  consistencies  were  adequate  (reported  in  subsequent 
sections). 
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The  Likert-scale  self-report  measure  was  originally  designed  with  a  pool  of  79  items. 
Sixty-three  items  directly  targeted  one  of  the  five  cultural  competence  factors  proposed  in  our 
model.  The  remaining  16  items  consisted  of  three  Cultural  Efficacy  items,  five  Learning 
Orientation  items,  and  an  eight-item  social  desirability  scale.  The  factor  analysis  was  conducted 
on  the  63  items  that  proposed  to  tap  factors  present  in  our  model.  Data  were  analyzed  with 
descriptive  statistics  and  exploratory  factor  analyses  as  sample  size  increased  with  each  data 
collection.  The  63  items  were  maintained  due  to  the  instability  of  factors  due  to  small  sample 
size  until  an  adequate  sample  size  of  approximately  400  subjects  was  reached. 

Prior  to  running  the  factor  analysis,  all  negatively-keyed  items  were  reverse  scored  and 
non-productive  items  were  eliminated  from  the  dataset  by  examining  means,  standard  deviations, 
minimums,  and  maximums.  We  adopted  three  criteria  for  defining  non-productive  items:  (1) 
very  high  means  (items  with  M  >  5  on  a  6-point  scale)  indicating  social  desirability,  (2)  small 
standard  deviations  (items  with  SD  <  1)  indicating  lack  of  variability,  and  (3)  range  restriction 
(items  whose  responses  did  not  encompass  the  entire  scale  from  1  to  6).  Items  that  met  two  or 
more  of  the  above  criteria  were  eliminated. 

Factor  analysis  with  oblique  direct  oblimin  rotation  was  used  to  examine  the  factor 
structure  of  the  self-report  questionnaire.  We  used  a  scree  plot  of  the  eigenvalues  to  guide 
multiple  exploratory  rotations  with  varying  number  of  dimensions  (Stevens,  2002).  The  scree 
plot  suggested  five  components  based  on  the  location  of  the  natural  bend  of  the  line.  Five  factors 
did  reveal  an  interpretable  and  reliable  factor  structure,  which  is  described  below.  The  number 
of  items  in  each  scale  was  detennined  based  on  adequate  reliability  (alpha  >  =  .70).  The  factor 
analysis  revealed  a  five-factor  structure  of  the  remaining  44  items  as  represented  in  Table  3. 

Each  of  the  factors  exhibited  adequate  reliability  (Cronbach  alpha  >  .70). 


Table  3 


Alphas  of  3 C  Factors 


Factor 

Cronbach’ s  a 

Number  of 
items 

Cultural  Interest 

0.73 

6 

Cultural  Relativism 

0.80 

10 

Cultural  Acuity 

0.70 

8 

Relationship  Orientation 

0.71 

7 

Interpersonal  Skills 

0.87 

13 

Overall 

0.91 

44 

The  factors  were  individually  interpreted  and  qualitatively  compared  to  the  hypothesized 
components.  The  factors  were  labeled  as  Cultural  Interest  (Cl),  Cultural  Relativism  (CR), 
Cultural  Acuity  (CA),  Relationship  Orientation  (RO),  and  Interpersonal  Skills  (IS).  Each  of 
these  factors  is  described  in  greater  detail  below.  Appendix  A  contains  the  complete  self-report. 
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Cultural  Interest.  This  factor  involves  a  Soldier’s  willingness  to  learn  about  and  engage 
with  the  local  populace  in  order  to  promote  mission  success.  A  comfort  with,  and  even 
enthusiasm  for,  the  uncertainty  that  coincides  with  these  interactions  also  is  a  critical  element  of 
this  ability.  A  common  thread  linking  the  items  on  the  Cultural  Interest  subscale  is  an  apparent 
willingness  to  voluntarily  identify  and  assume  additional  cross-cultural  responsibilities  to 
achieve  a  more  positive  end.  The  KSAAs  that  exhibit  particularly  good  fit  with  this  factor  are 
Dedication  to  one’s  mission  and  Openness  to  novel  situations.  This  factor  is  comprised  of  the 
following  items: 


Cultural  Interest 

1  •  Prior  to  a  deployment,  I  would  try  to  leam  the  basics  of  the  language  before  going,  whether 
directed  to  or  not. 

2-  If  I  knew  I  was  being  deployed,  I  would  spend  some  free  time  learning  about  the  cultural 
customs  before  I  left. 

3.  I  would  easily  and  believably  “fake  compassion”  with  foreign  citizens  to  achieve  the 
mission. 

4.  I  enjoy  making  sense  of  complex  situations. 

5.  Interacting  with  locals  in  order  to  build  relationships  during  deployments  is  worth  the  risks. 

6.  I  would  quickly  get  used  to  unfamiliar  customs  if  deployed. 

Cultural  Relativism.  Cultural  Relativism  refers  to  the  ability  to  recognize  that  cultures 
differ,  with  respect  to  individuals  who  come  from  a  background  different  than  one’s  own,  and  to 
understand  that  the  way  one’s  own  culture  operates  is  not  the  only  or  necessarily  the  best  way. 
Emotional  self-regulation  is  a  key  component  of  this  factor;  that  is,  the  ability  to  remain  flexible 
and  calm  in  different  cultural  contexts  when  initial  approaches/interactions  do  not  go  as  planned. 
The  common  threads  across  these  items  are  the  KSAAs  of  Flexibility  in  the  face  of  novel 
situations,  Tolerance  of  Uncertainty,  and  Openness  to  unfamiliar  practices.  This  factor  is 
comprised  of  the  following  items: 


Cultural  Relativism 

1 .  The  views  and  beliefs  of  American  culture  are  generally  superior  to  those  of  the 
countries  where  we  are  deployed  to  support. 

2.  Since  we  are  often  deployed  in  order  to  help  other  countries,  those  countries 
should  adjust  to  our  customs,  not  the  other  way  around. 

3.  Deployed  U.S.  forces  need  to  focus  less  on  compassion  and  more  on  “getting  the 
job  done”  when  dealing  with  locals. 

4.  If  I  find  a  common  practice  of  the  locals  offensive  while  deployed,  I  would  have 
trouble  understanding  why  the  locals  act  that  way. 

5.  I  find  the  thought  of  negotiating  with  foreign  village  elders  unpleasant. 

6.  As  an  American,  I  probably  do  not  have  as  many  biases  as  do  people  from 
Middle  Eastern  cultures. 

7.  If  you  know  the  basic  do’s  and  don’ts  of  a  country,  and  some  language,  that’s  all 
you  need  to  get  by  to  interact  with  locals  during  deployments. 
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8.  I  would  easily  change  my  outward  appearance  based  on  the  mission,  such  as 
switching  from  a  body  snatch  to  a  humanitarian  effort. 

9.  In  trying  to  persuade  a  village  elder  to  let  us  search  his  village,  I  would  probably 
fall  back  on  force  if  my  first  attempts  at  persuasion  did  not  work. 

10.  I  could  see  my  temper  getting  the  best  of  me  when  interacting  with 
unappreciative  foreign  citizens  during  a  deployment. 

Cultural  Acuity.  The  third  factor  was  labeled  Cultural  Acuity  because  it  is  comprised  of 
all  (but  one)  items  from  the  hypothesized  component  of  the  same  name.  This  factor  involves  the 
ability  to  fonn  accurate  cross-cultural  understandings  and  assessments  of  situational  dynamics, 
the  perspectives  of  others,  and  the  impact  of  cultural  actions  on  the  broader  mission.  KSAAs 
involved  in  this  factor  are  Perspective  Taking,  Sensemaking,  and  Big  Picture  Mentality.  One 
item  that  fell  under  this  factor  originated  from  Cognitive  Flexibility  (1  do  better  sticking  with  an 
approach  until  it  works  versus  changing  tactics )  also  seems  to  be  concerned  with  the  (in)ability 
to  make  an  accurate  assessment  of  a  situation  in  which  a  change  of  tactics  may  be  necessary. 

This  factor  comprises  the  following  items: 

Cultural  Acuity 

1 .  Without  the  help  of  fellow  Soldiers,  I  would  struggle  in  figuring  out  what  the  locals  are 
really  up  to  in  deployment  situations. 

2.  I  often  have  to  rely  on  others  to  adjust  my  perceptions  of  what  is  really  going  on  in  a  group 
setting. 

3.  I  often  have  trouble  envisioning  the  long-term  effects  of  my  actions. 

4.  It  would  be  hard  for  me  to  read  the  intent  of  a  foreign  citizen  with  whom  I  am 
communicating. 

5.  I  would  have  trouble  predicting  the  long-tenn  effects  of  my  actions  in  a  new  country. 

6.  When  watching  two  people  have  a  discussion,  I  can  pick  up  on  any  differences  between 
what  is  being  said  and  what  is  really  felt. 

7.  I  consider  myself  as  being  oblivious  to  what  is  really  going  on  in  group  interactions. 

8.  I  do  better  sticking  with  an  approach  until  it  works  versus  changing  tactics. 

Relationship  Orientation.  This  factor  was  termed  Relationship  Orientation  and  is 
concerned  with  the  general  tendency  to  value  personal  relationships  and  the  ability  to  feel  as 
others  feel  as  a  means  by  which  to  develop  genuine  relationships.  This  factor  is  largely 
comprised  of  the  KSAAs  of  Empathy,  Emotional  Self-Regulation,  and  Perspective  Taking  and  is 
focused  on  personal  relationships,  which  rely  on  an  accurate  understanding  of  and  reactions  to 
other  individuals.  This  factor  is  comprised  of  the  following  items: 

Relationship  Orientation 

1 .  I  often  “feel  the  pain”  of  others  when  someone  is  sharing  a  sad  story. 

2.  I  am  a  compassionate  and  trusting  person  in  general. 

3.  I  devote  significant  time  to  building  many  lasting  relationships  in  my  life. 

4.  I  get  upset  when  I  hear  people  making  fun  of  people  from  other  countries. 
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5.  If  a  trainee  was  resistant  to  my  instructions,  I  would  put  myself  in  their  shoes  to  figure  out 
why. 

6.  I  would  find  it  easy  to  be  casual  and  friendly  with  foreign  citizens  during  deployments. 

7.  I  sometimes  wonder  how  my  own  culture  influences  how  I  see  things. 

Interpersonal  Skills.  The  factor  of  Interpersonal  Skills  was  labeled  after  the  similar 
hypothesized  component  in  our  previous  model.  This  factor  is  comprised  of  all  but  one  item 
originating  from  that  component.  It  involves  the  ability  to  consistently  present  oneself  in  a 
manner  that  promotes  positive  short-  and  long-term  interactions  in  order  to  achieve  mission 
objectives.  The  KSAAs  associated  with  this  factor  are  Self-Monitoring  in  social  situations, 
Rapport  and  Relationship  Building  with  foreign  nationals,  and  Influence/Persuasion  over 
individuals.  One  item  that  loaded  on  this  factor  that  originated  from  the  hypothesized 
component  of  Cultural  Acuity  (/  would  have  little  problem  figuring  out  the  heart  of  the  matter 
when  observing  an  argument  between  U.S.  Soldiers  and  foreign  citizens )  is  also  concerned  with 
understanding  social  interactions  in  culturally  novel  environments.  Thus,  conceptually,  this  item 
appears  to  fit  relatively  well.  This  factor  is  comprised  of  the  following  items: 

Interpersonal  Skills 

1 .  I  can  win  over  a  group  of  strangers  with  ease. 

2.  I  use  my  sense  of  humor  to  quickly  put  people  at  ease. 

3.  My  personality  is  such  that  most  people  are  quickly  drawn  to  me. 

4.  I  possess  the  skills  needed  to  persuade  foreign  civilians  to  provide  sensitive  infonnation. 

5.  My  own  sense  of  humor  would  come  in  handy  during  deployments  to  put  foreign  locals 
at  ease. 

6.  On  a  deployment,  I  would  be  good  at  “working  the  locals”  to  give  me  needed  intel. 

7.  It  is  easy  for  me  to  quickly  gain  the  trust  of  others  through  casual  discussion. 

8.  Negotiating  with  village  leaders  during  a  deployment  would  fit  my  abilities. 

9.  I  am  good  at  getting  others  to  see  my  point  of  view. 

10.  I  would  have  little  problem  figuring  out  the  heart  of  the  matter  when  observing  an 
argument  between  U.S.  Soldiers  and  foreign  citizens. 

1 1 .  My  personality  is  such  that,  in  a  foreign  country,  I  could  quickly  put  an  irate  citizen  at 
ease. 

12.  I  would  probably  rely  on  another  team  member  to  strike  up  initial  conversations  with 
foreign  citizens  when  deployed,  as  this  is  not  my  strong  suit. 

13.  I  would  befriend  locals  during  deployments  to  support  mission  success. 

Developmental  Levels 

In  addition  to  the  changes  in  the  factors  of  the  3C  model  described  above,  the 
developmental  levels  of  the  model  were  modified  also.  The  original  model  of  3C  that  evolved 
within  this  effort  depicted  four  levels  of  development  (McCloskey,  Behymer,  Papautsky,  Ross, 

&  Abbe,  2010).  However,  the  simulation  interviews  conducted  during  this  effort  suggested  that 
the  addition  of  a  fifth  competence  level  (i.e.,  separating  the  Intermediate  level  into  two  levels — 
Beginner  and  Intermediate)  might  improve  the  utility  of  the  model.  After  coding  the  simulation 
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interviews  for  both  the  presence  of  individual  competencies  and  one  overall  competence  level, 
the  researchers  discovered  that  58  out  71  participants  (82%)  were  rated  in  the  two  intennediate 
competence  levels.  Rather  than  recode  the  interviews  with  different  criteria  for  each  of  the  four 
levels,  the  research  team  added  the  fifth  developmental  level. 

Only  two  of  the  simulation  interview  participants  were  rated  at  the  lowest  level  of 
competence  (i.e.,  Precompetent).  Although  this  might  initially  suggest  that  this  developmental 
level  be  eliminated  from  the  model,  data  from  the  team  ranking  task  interview  suggested 
otherwise.  In  conducting  team  ranking  interviews  where  participants  described  the  perfonnance 
and  general  characteristics  of  recent  team  members  in  deployment  settings,  nearly  90%  of 
participants  recalled  team  members  who  exhibited  strong  Precompetent  traits  and  rated  them 
very  low  on  a  competence  scale.  Participants  provided  more  than  250  descriptors  of  low- 
competence  Soldiers  on  their  teams,  referring  to  them  mainly  as  having  strong  negative  attitudes 
toward  foreign  nationals  and  an  active  avoidance  toward  any  interactions,  even  if  they  supported 
the  mission.  This  suggests  that,  while  our  simulation  interview  might  not  be  conducive  to 
eliciting  precompetent  responses,  these  Soldiers  are  not  as  rare  as  the  data  suggest.  Therefore, 
we  retained  the  Precompetent  level  in  our  developmental  model.  An  in-depth  description  of  each 
level  of  competence  follows. 

(1)  Precompetent 

Soldiers  at  the  Precompetent  level  of  3C  often  do  not  recognize  the  mission  value  in 
interacting  with  locals  beyond  what  is  ordered.  During  deployments,  they  tend  to  actively  avoid 
contact  with  the  local  populace  whenever  possible,  especially  during  free  time.  Soldiers  at  this 
level  may  also  get  frustrated  easily  during  required  interactions  and  may  have  little  desire  to  hide 
this  frustration.  They  are  often  uncomfortable  with  the  uncertainty  involved  with  cross-cultural 
interactions  and  have  little  regard  for  the  customs  of  the  countries  they  are  operating  in. 

(2)  Beginner 

The  second  level  is  Beginner.  Soldiers  at  the  Beginner  level  of  cross-cultural  competence 
have  some  initial  knowledge  of  the  customs  of  the  area  they  will  be  operating  in,  and  they  use 
this  basic  list  of  rules  and  taboos  to  guide  their  interactions.  These  Soldiers  all  have  at  least  a 
minimum  willingness  to  engage  with  the  local  populace  in  order  to  support  the  mission,  but  they 
typically  do  not  recognize  the  benefits  that  cross-cultural  interactions  can  offer  to  mission 
success.  Beginners  often  view  all  people  as  the  same  and,  therefore,  they  will  have  trouble 
understanding  what  locals  may  really  be  thinking  or  feeling.  They  may  have  basic  interpersonal 
skills,  but  without  an  understanding  of  their  own  cultural  influences,  they  may  struggle  in  using 
these  skills  in  cross-cultural  settings. 

Originally,  the  Beginner  level  was  a  part  of  the  Intermediate  level  and  it  included  both 
those  Soldiers  who  were  pure  novices  to  cross-cultural  development  and  those  with  some  initial 
basic  skills  based  on  experience.  This  Beginner  level  now  encompasses  just  those  who  are  at  the 
pure  novice  level;  those  Soldiers  who  demonstrate  fundamental,  experience-based  competence 
are  now  represented  within  the  Intennediate  level  of  development. 

(3)  Intennediate 
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Soldiers  at  the  Intermediate  level  of  cross-cultural  competence  typically  have  a  basic 
awareness  of  the  importance  of  cross-cultural  knowledge  and  interactions  in  supporting  mission 
success.  They  also  are  willing  to  make  limited  efforts  to  engage  with  the  local  populace  and 
leam  about  the  culture  in  order  to  do  their  jobs  better.  These  Soldiers  tend  to  have  basic 
relationship-building  and  persuasion  skills  as  well.  Areas  where  these  Soldiers  may  struggle 
tend  to  be  in  their  openness  to  extreme  cultural  differences  and  in  accuracy  in  their  attempts  to 
assume  the  perspectives  of  locals.  They  also  may  struggle  in  adjusting  their  cross-cultural 
perceptions  or  attitudes  once  they  are  formed.  Intermediate  level  leaders  take  responsibility  for 
the  cultural  interactions  of  their  Soldiers  and  provide  effective  guidance  on  how  to  avoid  most 
cross-cultural  incidents,  but  they  are  not  typically  proactive  in  shaping  the  cultural  environment. 
They  will  do  no  harm,  but  also  may  not  have  the  skills  to  take  initiative  in  cross-cultural 
encounters. 

(4)  Proficient 

Soldiers  at  the  Proficient  level  of  cross-cultural  competence  are  adept  at  controlling  their 
emotions  during  tense  or  uncertain  cross-cultural  encounters.  They  understand  that  their 
interactions  can  have  a  broad  and  long-term  impact  on  mission  success,  especially  in  deployment 
settings.  They  tend  to  remain  flexible  in  their  actions  with  and  attitudes  toward  foreign 
populations,  allowing  them  to  adjust  their  perceptions  when  new  information  is  available.  These 
Soldiers  also  recognize  their  own  cultural  biases  and  how  they  influence  their  own  perceptions. 
However,  culturally  proficient  Soldiers  still  have  room  for  development.  Although  they  have 
generally  decent  abilities  to  assume  the  perspectives  of  others  and  to  assess  the  long-term  effects 
of  their  actions  on  the  cultural  aspects  of  the  mission,  the  depth  and  accuracy  of  their 
understanding  can  be  improved  by  including  cultural  elements  in  their  assessments. 

(5)  Advanced: 

Soldiers  at  the  Advanced  level  of  cross-cultural  competence  can  typically  adjust  to  new 
cultural  environments  easily  with  little  advance  preparation  required.  They  are  ultimately 
comfortable  with  the  uncertainty  of  new  environments  and  are  highly  skilled  at  both  assessing 
the  cultural  terrain  and  using  their  assessments  to  successfully  control  the  course  of  all  their 
cross-cultural  encounters.  They  have  a  clear  awareness  of  their  own  cultural  biases  as  well  as  a 
strong  ability  to  predict  both  the  short-  and  long-tenn  effects  of  their  cultural  interactions  on 
mission  objectives. 

Assessment  Battery  Validation  and  Refinement 

Once  our  model  of  3C  was  finalized,  the  next  step  was  to  refine  our  metrics  to  measure 
the  components  of  the  model.  To  accomplish  this,  we  sought  to  establish  both  the  reliability  and 
validity  of  the  Cross-Cultural  Assessment  Tool  (C-CAT). 

Reliability.  To  examine  the  reliability  of  the  assessment  battery,  the  research  team 
utilized  the  pre-deployment  and  post-deployment  cadet  data,  because  most  of  the  same  measures 
were  administered  on  two  occasions.  Pearson  correlation  coefficients  were  calculated  to 
evaluate  test-retest  reliability  (Cohen,  Cohen,  West,  &  Aiken,  2003)  for  the  self-report  factors  as 
well  as  the  SJT  and  vignettes  that  were  common  across  the  testing  occasions.  The  team  found 
significant  positive  correlations  between  pre-  and  post-test  on  each  factor  (Table  4),  with  the 
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strongest  relationship  for  Interpersonal  Skills  at  r  =  .74,  p  <  .01.  In  addition,  significant 
correlations  were  present  for  each  of  the  SJTs  and  vignettes  used  in  both  the  pre-  and  post-  test 
conditions  (SJT1,  Vignette  3,  Meeting  With  Leaders,  and  Vignettes  4  and  5,  Burn  Victim 
Support  and  Needs  Assessment  Gone  Wrong,  which  had  the  same  questions  with  a  different 
cover  story).  The  significant  correlations  are  of  moderate  (rather  than  high)  strength  as 
hypothesized,  because  cadets’  experiences  in  the  cultural  immersion  program  resulted  in  an 
increase  in  overall  3C  as  reflected  by  higher  ratings  on  the  self-report  measure.  These  findings 
support  the  consistency  of  responses  to  the  set  of  measures. 

Table  4 

Test-Retest  Reliability  of  Cadet  Data _ 


Reliability 


Cultural  Interest 

r 

0.66 

N 

143 

Cultural  Relativism 

r 

0.63 

N 

143 

Cultural  Acuity 

r 

0.48 

N 

143 

Relationship 

r 

0.63 

Orientation 

N 

143 

Interpersonal  Skills 

r 

0.74 

N 

143 

All  Items 

r 

0.59 

N 

171 

SJT1 

r 

0.41 

N 

142 

Vignette  3  (Meeting 

r 

0.48 

With  Leaders) 

N 

138 

Vignette  4  and  5 

r 

0.41 

(Burn  Victim 

Support/Needs 

N 

135 

Assessment  Gone 

Wrong) 

Note:  All  significant  at  p  <  .001. 


Criterion  Related  Validity.  To  establish  criterion  validity,  the  research  team  examined 
the  relationships  between  self-reports  on  the  five  factors  and  supervisor  and  peer  ratings.  As 
discussed  in  the  Methods  section,  supervisors  and  peers  rated  the  cadets  on  observable  behaviors 
that  supported  the  training  objectives  established  by  Cadet  Command.  Significant  correlations 
were  found  between  both  Cultural  Interest  and  Interpersonal  Skills  and  supervisor  ratings  for  the 
pre-test  at  r  =  .22,  p  =  .03  and  r  =  25, p  =  .01,  respectively.  In  addition,  significant  correlations 
were  found  between  Cultural  Interest  and  peer  ratings  and  between  Interpersonal  Skills  and 
supervisor  ratings  for  the  post-test  at  r  =  2\,p  =  .01  and  r  =  .20,  p  =  .03,  respectively  (see  Table 
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5  for  all  correlation  coefficients).  These  findings  are  consistent  with  our  expectations  that  the 
Cultural  Interest  factor  is  comprised  of  items  measuring  affective  skills  and  the  Interpersonal 
Skills  factor  targets  behaviors,  both  of  which  are  observable  and  can  be  evaluated  by  an  outside 
party,  such  as  a  supervisor.  As  only  one  factor  was  significantly  correlated  with  ratings,  it  is 
possible  that  peers  may  not  have  had  the  level  of  experience  needed  to  assess  their  colleagues. 
Although  our  findings  suggest  some  evidence  of  criterion  validity  of  the  Self-Report  measure, 
criterion  validity  was  not  fully  established,  possibly  due  to  the  limitations  of  our  criterion 
measures.  Three  limitations  in  these  measures  resulted  from  a  convenience  sample:  (1)  ratings 
were  based  on  cadets,  who  are  significantly  less  experienced  than  Soldiers,  (2)  ratings  were 
based  on  a  2-week  long  cross-cultural  experience,  which  may  be  too  short  for  supervisors/peers 
to  accurately  evaluate  performance,  and  (3)  the  activities  that  cadets  were  involved  in  differed 
(and  were  limited)  by  country  of  destination,  complicating  the  comparison  across  groups.  Thus, 
the  reported  ratings  are  not  adequate  to  accurately  reflect  cadet  3C.  Provided  a  sample  of 
Soldiers  who  perform  in  the  field  for  prolonged  periods  of  time,  supervisors  and  peers  would  be 
able  to  more  accurately  rate  them  based  on  3C. 

Table  5 

Correlation  for  Self-Report  and  Supervisor/Peer  Rating  for  Criterion  Validation 


Pre-test 

Post-test 

Supervisor  Peer 
rate  rate 

Supervisor  Peer 
rate  rate 

Cultural  Interest 

r 

0.21* 

0.14 

0.14 

0.21*= 

Sig. 

0.03 

0.09 

0.14 

0.01 

N 

105 

156 

118 

161 

Cultural  Relativism 

r 

-0.01 

0.03 

-0.09 

0.10 

Sig. 

0.91 

0.71 

0.35 

0.19 

N 

105 

156 

118 

161 

Cultural  Acuity 

r 

0.05 

-0.08 

-0.06 

0.02 

Sig. 

0.63 

0.32 

0.54 

0.80 

N 

105 

156 

118 

161 

Relationship 

r 

0.14 

0.03 

0.08 

0.13 

Orientation 

Sig. 

0.14 

0.67 

0.41 

0.11 

N 

105 

156 

118 

161 

Interpersonal  Skills 

r 

0.25** 

0.02 

0.20* 

0.13 

Sig. 

0.01 

0.84 

0.03 

0.10 

N 

105 

156 

118 

161 

All  Items 

r 

0.18 

0.03 

0.03 

0.07 

Sig. 

0.07 

0.72 

0.74 

0.39 

N 

105 

156 

123 

174 

*p  <  .05,  **p  <  .01 


A  series  of  correlations  were  also  conducted  between  all  SJTs  and  vignettes  and  peer  and 
supervisor  ratings  to  assess  the  extent  to  which  performance  on  our  scenario-based  measures 
correlated  with  actual  performance  as  assessed  by  peers  and  supervisors.  Vignette  1  (The  New 
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Interpreter)  and  SJT2  (Looking  for  Weapons)  showed  a  pattern  of  significant  correlations  and 
were  thus  examined  further.  A  pattern  of  findings  suggest  that  Vignette  1.4  (the  4th  item  in  The 
New  Interpreter  scenario)  is  significantly  correlated  with  the  criterion  measure  of  Supervisor 
Rate  (r(100)  =  .24,  /;  =  .01)  and  multiple  Supervisor  Rate  items.  Further  investigations  revealed 
that  an  average  of  Vignettes  1.3,  1.4,  and  1.5  is  also  significantly  correlated  with  Supervisor  Rate 
(r(100)  =  .25, p  =  .01).  Both  SJT  2  average  and  SJT  2.1  are  significantly  correlated  with  Peer 
Rate  Average  (r(  156)  =  .16,/;=  .05  and  r(126)  =  .21,/;=  .01,  respectively)  and  multiple  Peer 
Rate  items.  As  a  result  of  this  analysis,  SJT  2  (Looking  for  Weapons)  and  Vignette  1  (The  New 
Interpreter)  were  included  in  the  final  battery  as  they  showed  the  most  promise,  and  eliminating 
the  other  SJTs  and  vignettes  from  the  battery  shortened  the  time  to  take  the  battery  without 
reducing  the  effectiveness  of  the  battery. 

Construct  Validity.  The  correlations  between  the  five  factors  are  shown  in  Table  6.  All 
correlations  are  significant  atp  <  .001,  N=  369. 

Table  6 

Correlations  for  the  Five  Factors 


Correlations 


Cultural 

Interest 

Cultural 

Relativism 

Cultural 

Acuity 

Relationship 

Orientation 

Interpersonal 

Skills 

Cultural 

Interest 

Pearson 

correlation 

1 

.348 

.388 

.431 

.612 

Cultural 

Relativism 

Pearson 

correlation 

.348 

1 

.338 

.344 

.281 

Cultural 

Acuity 

Pearson 

correlation 

.388 

.338 

1 

.275 

.466 

Relationship 

Orientation 

Pearson 

correlation 

.431 

.344 

.275 

1 

.422 

Interpersonal 

Skills 

Pearson 

correlation 

.612 

.281 

.466 

.422 

1 

Using  the  refined  self-report  factors  and  the  SJT  and  vignette  identified  above  as  being 
significantly  correlated  with  peer  and  supervisor  ratings,  we  explored  the  extent  to  which  the 
measures  are  targeting  similar  constructs  (convergent  validity).  The  goal  was  also  to  examine 
which  factor  each  SJT/vignette  question  seemed  to  tap  into  the  most.  As  the  self-report  factors 
were  initially  developed  and  further  refined  to  tap  into  very  specific  aspects  of  3C,  we  calculated 
Pearson  correlation  coefficients  between  these  factors  and  SJT2  (Looking  for  Weapons)  and 
Vignette  1  (The  New  Interpreter)  individual  items.  The  overall  pattern  of  results  revealed  that 
multiple  SJT  and  vignette  items  were  significantly  correlated  with  self-report  factors  (Table  7); 
however,  these  significant  correlations  may  be  due  to  the  large  sample  size  because  the  size  of 
the  correlation  is  small. 
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Table  7 

Correlation  Coefficients  for  Self-Report  Factors,  Situational  Judgment  Test  2,  and  Vignette  1 


SJT 

SJT 

SJT 

SJT 

SJT 

2.1 

2.2 

2.3 

2.4 

2.5 

VI. 1 

VI. 2 

VI. 3 

VI. 4 

VI. 5 

Cultural  Interest 

r 

0.02 

0.03 

0.06 

0.14** 

0.03 

0.18** 

0.13* 

0.15** 

0.11* 

0.10 

Sig 

0.66 

0.49 

0.24 

0.00 

0.61 

0.00 

0.01 

0.00 

0.03 

0.05 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

Cultural 

r 

-0.07 

-0.05 

0.15** 

0.10* 

0.00 

0.12* 

0.11* 

0.19** 

0.26** 

0.18** 

Relativism 

Sig 

0.16 

0.28 

0.00 

0.04 

0.92 

0.02 

0.04 

0.00 

0.00 

0.00 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

Cultural  Acuity 

r 

-0.03 

-0.02 

0.03 

0.12* 

-0.03 

0.05 

0.07 

0.17** 

0.07 

0.03 

Sig 

0.61 

0.68 

0.51 

0.02 

0.51 

0.36 

0.16 

0.00 

0.20 

0.62 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

Relationship 

r 

-0.14 

-0.21 

0.07 

-0.03 

-0.20 

0.15** 

0.21** 

0.20** 

0.29** 

0.21** 

Orientation 

Sig 

0.01 

0.00 

0.19 

0.55 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

Interpersonal 

r 

0.06 

-0.01 

0.13** 

0.13** 

-0.03 

0.20** 

0.16** 

0.19** 

0.09 

0.08 

Skills 

Sig 

0.23 

0.92 

0.01 

0.01 

0.53 

0.00 

0.00 

0.00 

0.08 

0.14 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

All  SR  Items 

r 

-0.03 

-0.07 

0.14** 

0.13** 

-0.06 

0.20** 

0.19** 

0.25** 

0.23** 

0.16** 

Sig 

0.50 

0.18 

0.01 

0.01 

0.25 

0.00 

0.00 

0.00 

0.00 

0.00 

N 

400 

401 

400 

400 

400 

376 

377 

376 

376 

376 

*p  <  .05,  **p  <  .01  -  in  positive  direction 


Final  Assessment  Battery 

Based  on  the  results  described  above  the  Final  Assessment  Battery  consisted  of  the 
following: 

•  44  self-report  items 

Based  on  elimination  of  non-productive  items  and  subsequent  exploratory  factor 
analysis. 

•  Vignette  1  (The  New  Interpreter) 

Statistically  and  conceptually  consistent  with  supervisor  ratings. 

•  SJT  2  (Looking  for  Weapons) 

Statistically  and  conceptually  consistent  with  peer  ratings. 
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Battery  Scoring 

The  output  of  the  battery  consists  of  a  score  on  each  of  the  five  factors  and  also  provides 
the  user  with  an  overall  assessment  of  his  or  her  3C.  Each  vignette  and  SJT  question  contributes 
to  a  single  factor  based  on  the  following  process.  First,  we  examined  the  correlations  between 
self-report  factors  and  each  SJT/vignette  question.  Because  some  of  the  questions  were  not 
significantly  correlated  with  a  specific  self-report  factor  or  were  significantly  correlated  with 
multiple  self-report  factors,  two  members  of  the  research  team  independently  assessed  which 
factor  each  question  most  highly  related  to  based  on  a  process  that  involved  both  raters 
reviewing  the  statistical  results  along  with  the  original  factor  the  question  was  designed  to  assess 
and  then  placing  them  within  a  factor  category.  The  raters  then  met  to  discuss  rating  disparities, 
with  a  third  research  team  member  involved  to  resolve  any  remaining  disagreements.  Based  on 
these  results,  each  SJT/vignette  question  was  assigned  to  a  specific  factor  as  shown  in  Table  8. 
The  limitations  of  this  approach  are  discussed  in  the  Limitations  and  Future  Research  section. 
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Table  8 


Factor  Assignments  ofSJTs  and  Vignettes 


Question 

Correlation 

coefficient 

Raters 

Final  factor 

V 1 . 1  How  would  you  feel  at  the 
moment  (when  interpreter  is  late)? 

n.s. 

Cultural 

Relativism 

Cultural 

Relativism 

V 1 .2  Assuming  that  the  lateness  issue  is 
resolved,  what  would  you  say  next  to 
your  interpreter? 

0.21** 

Relationship 

Orientation 

Relationship 

Orientation 

Relationship 

Orientation 

V 1 .3  What  would  you  do  now  in  terms 
of  the  meeting?  (last  minute  notice, 
little  preparation  time) 

0.19** 

Cultural 

Relativism 

Cultural 

Relativism 

Cultural 

Relativism 

V 1 .4  What  would  your  thoughts  be 
toward  the  interpreter  now? 

0.29** 

Relationship 

Orientation 

Cultural 

Acuity 

Cultural 

Acuity 

VI  .5  What  do  you  say  or  do  now  in 
response  to  the  interpreter? 

0.18**  Cultural 
Relativism 

Cultural 

Relativism 

Cultural 

Relativism 

SJT2. 1  As  the  platoon  leader  in  this 
situation,  what  immediate  CO  As  would 
you  take?  (men  approaching  vehicles) 

n.s. 

Interpersonal 

Skills 

Interpersonal 

Skills 

SJT2.2  What  questions  would  you  ask 
these  men? 

n.s. 

Cultural 

Acuity 

Cultural 

Acuity 

SJT2.3  As  the  platoon  leader  in  this 
situation,  what  immediate  CO  As  would 
you  take?  (leader  forbids  search,  men 
run  off) 

0.15** 

Cultural 

Relativism 

Relationship 

Orientation 

Relationship 

Orientation 

SJT2.4  As  the  platoon  leader  in  this 
situation  what  immediate  COAs  would 
you  take? 

0.14** 

Cultural 

Interest 

Cultural 

Interest 

Cultural 

Interest 

SJT  2.5  As  the  platoon  leader  in  this 
situation,  what  immediate  COAs  would 
you  take?  (families  gather,  don’t  want 
to  frighten  them  with  search) 

n.s 

Cultural 

Acuity 

Cultural 

Acuity 

**p  <  .05  -  in  positive  direction 


To  illustrate  how  a  factor  score  was  calculated,  we  provide  the  following  example.  The 
Cultural  Interest  score  is  based  on  that  person’s  responses  to  the  six  self  report  questions 
associated  with  Cultural  Interest  and  Question  #4  from  the  SJT.  To  determine  this  score,  z- 
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scores  for  the  average  of  the  self-report  questions  and  the  SJT  questions  were  calculated.  In 
these  calculations,  self-report  factors  were  weighted  by  75%  as  they  were  explicitly  designed  to 
target  3C.  The  remaining  25%  was  distributed  across  the  SJT  and  vignette  items.  For  example, 
if  an  equation  consisted  of  one  SJT  item,  it  received  a  25%  weight,  but  if  it  consisted  of  two 
items,  they  received  12.5%  each.  Thus,  the  weighting  of  scenario-based  measures  was 
consistently  25%  regardless  of  how  many  items  were  used.  This  breakdown  was  derived  from 
the  methodical  development  of  the  self-report  and  subsequently  a  clear  measure  of  3C  reflected 
by  the  higher  weighting.  The  scenario-based  measures  have  more  face  validity,  but  require 
further  data  collections  and  analyses,  to  achieve  full  understanding  of  their  contribution.  Thus, 
for  example,  the  overall  score  for  Cultural  Interest  is  calculated  by  the  following  formula: 
(SRAvgzscore*.75  +  SJTQ4zscore*.25).  This  score  places  the  individual  into  one  of  the  six 
categories  shown  in  Table  9.  These  categories  were  developed  in  order  to  provide  clear  feedback 
to  the  user  on  their  performance  once  they  had  completed  the  battery. 

Table  9 


Factor  categories 


Low 

Moderate  High 

Below  -1  -1  SD  to  -Vi 

-Vi  SD  to 

Mean  to  Vi  Vi  SD  to  1  SD  Above  1  SD 

SD  SD 

Mean 

SD 

The  formulas  for  each  factor  are  shown  in  Table  10  (all  variables  are  z-scores). 

Table  10 

Formulas  per  factor 

Factor 

Formula 

Cultural  Interest 

(SRAvg*.75)  +  (SJTQ4*.25) 

Cultural  Relativism 

(SRAvg*.75)  +  (VigQl*.083)  +  (VigQ3*.083) 
(VigQ5*.083) 

Cultural  Acuity 

(SRAvg*.75)+  (VigQ4*.083)  +  (SJTQ2*.083) 

+  (SJTQ5*.083) 

Relationship  Orientation 

(SRAvg*.75)  +  (VigQ2*.125)  +  (SJTQ3M25) 

Interpersonal  Skills 

(SRAvg*.75)  +  (SJTQ1*.25) 

The  relative  weightings  of  self-report  factors  to  be  used  in  calculating  an  overall  score 
were  determined  by  both  statistical  and  theoretical  approaches.  We  first  examined  which  factors 
were  higher  predictors  of  peer  and  supervisor  ratings  and  let  this  infonnation  guide  the 
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determination  of  relative  weightings  accordingly.  Based  on  qualitative  analyses  that  informed 
new  factor  labels,  we  determined  that  each  factor  was  associated  with  affective,  cognitive,  or 
behavioral  skills.  Cultural  Interest  and  Relationship  Orientation  factors,  for  instance,  target  the 
experience  of  feelings  and  emotions,  and  are  thus  affective  in  nature.  Cultural  Relativism  and 
Cultural  Acuity  factors  are  both  cognitive,  as  items  are  concerned  with  decision-making 
processes.  Lastly,  the  Interpersonal  Skills  factor  is  behavioral  in  that  its  items  reflect  actions  that 
are  easily  observed  by  others.  Given  that  our  research  findings  suggested  that  cross-cultural 
competence  comprises  a  combination  of  affective,  cognitive,  and  behavioral  skills,  we  weighted 
each  set  equally.  Specifically,  the  weightings  are  reported  in  Table  11. 

Table  11 

Weightings  of  Each  Self-Report  Factor  for  Feedback  Equations 


Weight  Total 

Affective  skills 

Cultural  Interest 

23% 

33% 

Relationship  Orientation 

10% 

Cognitive  skills 

Cultural  Relativism 

13% 

33% 

Cultural  Acuity 

21% 

Behavioral  skills 

Interpersonal  Skills 

33% 

33% 

Total 

100% 

Hence,  to  calculate  an  overall  competence  score  the  following  formula  was  used:  Overall 
score  =  Cultural  Interest  (.23)  +  Relationship  Orientation  (.10)  +  Cultural  Relativism  (.13)  + 
Cultural  Acuity  (.21)  +  Interpersonal  Skills  (.33). 

To  combine  the  different  measures  (self-report,  SJTs,  and  vignettes)  into  a  single  score,  it 
was  necessary  to  standardize  an  individual’s  score  on  each  measure  by  transforming  it  into  a  z- 
score.  The  cutoff  for  each  level  of  competence  was  not  based  on  assigning  a  specific  percentage 
of  participants  to  each  level  of  competence.  Rather,  to  determine  how  responses  on  the 
assessment  battery  would  translate  into  overall  competence  ratings,  the  research  team  performed 
a  qualitative  analysis,  comparing  z-scores  with  individual  factor  ratings  for  369  participants.  The 
researchers  identified  logical  breakpoints  for  the  z-scores  based  on  the  qualitative  descriptions  of 
each  of  the  developmental  levels.  For  example,  in  looking  at  participants’  factor  scores,  starting 
with  those  participants  who  received  the  lowest  z-scores  on  the  overall  assessment,  a  breakpoint 
in  z-score  occurred  at  approximately  -0.869.  Participants  who  scored  below  this  level  all  scored 
low  on  the  affective  factors  of  Cultural  Sensitivity  and  Cultural  Interest,  factors  that  our  model 
suggest  distinguish  the  Precompetent  Soldier  from  the  Beginner. 

The  team  used  similar  reasoning  methods  to  develop  z-score  cutoffs  for  each  of  the 
overall  competence  levels.  The  cutoffs  used  in  the  assessment  tool  (see  Table  12)  resulted  in 
more  than  half  of  the  Soldiers  in  our  sample  being  placed  into  the  Intermediate  level  or  lower, 
but  fewer  than  10%  were  placed  in  the  Precompetent  level.  Fifteen  percent  of  our  sample 
received  the  highest  rating  of  Advanced.  This  distribution  is  consistent  with  our  team’s 
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qualitative  assessment  of  where  our  operational  forces  would  tend  to  fall  on  the  3C  continuum 
based  on  our  interviews  collected  and  perceptions  formed  to  date  based  on  the  results  of  the  team 
ranking  task  and  interaction  with  3C  trainers.  We  do  not  believe  that  an  equal  number  of 
Soldiers  will  be  at  each  level  of  development,  and  to  force  the  coding  scheme  to  represent  this 
artificial  distribution  of  competence  levels  would  be  inaccurate.  Furthermore,  feedback  has  been 
structured  to  emphasize  that,  while  there  will  always  be  areas  for  improvement  in  3C  regardless 
of  overall  score  and  factor  scores,  the  scores  received  are  only  indirect  indicators  of 
competencies  based  on  the  responses  provided. 


Table  12 


Ranges  for  Each  Level  of  Competence 


Level 

z-score  Range 

%  of  participants  at 
level 

Precompetent 

<  -0.869 

9.2% 

Beginner 

-0.868-0.491 

13.3% 

Intermediate 

-0.490-0.162 

34.4% 

Proficient 

0.163-0.625 

27.9% 

Advanced 

>  0.626 

15.2% 

The  Computer  Based  Assessment  Battery 

The  finalized  battery  was  computerized  using  Adobe  Flash.  The  computerized  tool  can 
be  used  online  or  offline  and  is  designed  to  provide  immediate  feedback  (which  can  be  printed 
out)  to  the  user.  The  computer  based  battery  will  operate  as  follows  (see  Appendix  E  for 
screenshot  examples).  First,  participants  view  a  screen  stating  the  purpose  of  the  battery  and  are 
provided  instructions  on  how  to  take  the  battery.  Next,  participants  answer  the  44  self  report 
items,  which  have  been  divided  into  segments  of  seven  to  eight  questions.  Participants  are 
provided  with  a  progress  bar  so  they  are  aware  of  how  much  longer  they  have  until  they  have 
completed  the  battery.  Participants  are  then  provided  with  instructions  and  an  example  of  how  to 
complete  the  SJT.  Validation  checking  is  in  place  so  participants  cannot  proceed  until  they  have 
provided  a  valid  answer  to  each  SJT  question.  This  process  is  then  repeated  for  the  vignette. 

Upon  completion,  participants  are  provided  feedback  in  a  report  form  that  provides  an 
overall  competence  score,  scores  on  each  of  the  five  factors,  and  feedback  for  improving  their 
3C.  A  print  button  provides  easy  access  to  a  hard  copy  of  the  report.  The  report  begins  by 
providing  the  individual  with  an  overall  score  of  3  C  on  a  scale  ranging  from  Precompetent  to 
Advanced.  A  graphic  representing  their  score  on  each  of  the  five  factors  and  instructions  on  how 
to  interpret  the  scores  are  also  provided. 

Regardless  of  their  individual  scores,  the  report  provides  a  description  of  each  level  of 
competence.  The  descriptions  within  the  report  are: 

Soldiers  at  the  PRECOMPETENT  level  of  cross-cultural  competence  often  do  not 
recognize  the  mission-value  in  interacting  with  locals  beyond  what  is  ordered.  During 
deployments,  they  tend  to  actively  avoid  contact  with  the  local  populace  whenever  possible, 
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especially  during  free  time.  Soldiers  at  this  level  may  also  get  frustrated  easily  during  required 
interactions,  and  may  have  little  desire  to  hide  this  frustration.  They  are  often  uncomfortable 
with  the  uncertainty  involved  with  cross-cultural  interactions  and  have  little  regard  for  the 
customs  of  the  countries  they  are  operating  in. 

Soldiers  at  the  BEGINNER  level  of  cross-cultural  competence  will  have  some  initial 
knowledge  of  the  customs  of  the  area  they  will  be  operating  in,  and  they  will  use  this  basic  list  of 
rules  and  taboos  to  guide  their  interactions.  These  Soldiers  all  have  at  least  a  minimum 
willingness  to  engage  with  the  local  populace  in  order  to  support  the  mission,  but  they  typically 
do  not  recognize  the  benefits  that  cross-cultural  interactions  can  offer  to  mission  success. 
Beginners  often  view  all  people  as  the  same  and,  therefore,  they  will  have  trouble  understanding 
what  locals  may  really  be  thinking  or  feeling.  They  may  have  basic  interpersonal  skills,  but 
without  an  understanding  of  their  own  cultural  influences,  they  may  struggle  in  using  these  skills 
in  cross-cultural  settings. 

Soldiers  at  the  INTERMEDIATE  level  of  cross-cultural  competence  usually  have  a  basic 
awareness  of  the  importance  of  cross-cultural  knowledge  and  interactions  in  supporting  mission 
success.  They  also  are  willing  to  make  efforts  to  engage  with  the  local  populace  and  leam  about 
the  culture  in  order  to  do  their  jobs  better.  These  Soldiers  tend  to  have  basic  relationship¬ 
building  and  persuasion  skills  as  well.  Areas  where  these  Soldiers  may  struggle  tend  to  be  in 
their  openness  to  extreme  cultural  differences,  and  in  accuracy  of  their  attempts  to  assume  the 
perspectives  of  locals.  They  also  may  struggle  in  adjusting  their  cross-cultural  perceptions  or 
attitudes  once  they  are  fonned.  Intermediate  level  leaders  take  responsibility  for  the  cultural 
interactions  of  his/her  Soldiers,  and  provide  effective  guidance  on  how  to  avoid  cross-cultural 
incidents. 

Soldiers  at  the  PROFICIENT  level  of  cross-cultural  competence  are  skilled  at  controlling 
their  emotions  during  tense  or  uncertain  cross-cultural  encounters.  They  also  understand  that 
their  interactions  can  have  a  broad  and  long-tenn  impact  on  mission  success,  especially  in 
deployment  settings.  They  tend  to  remain  flexible  in  their  actions  with,  and  attitudes  toward 
foreign  populations,  allowing  them  to  adjust  their  perceptions  when  new  infonnation  is  available. 
These  Soldiers  also  recognize  their  own  cultural  biases  and  how  they  influence  their  own 
perceptions.  The  culturally  proficient  Soldier  still  has  room  for  development,  however.  While 
they  have  generally  decent  abilities  to  assume  the  perspectives  of  others,  and  assess  the  long¬ 
term  effects  of  their  actions  on  the  cultural  aspects  of  the  mission,  the  depth  and  accuracy  of  their 
understanding  can  be  improved  by  including  cultural  elements  in  their  assessments. 

Soldiers  at  the  ADVANCED  level  of  cross-cultural  competence  can  adjust  to  new  cultural 
environments  easily,  with  little  advance  preparation  required.  They  are  ultimately  comfortable 
with  the  uncertainty  of  new  situations  and  people,  and  are  highly  skilled  at  both  assessing  the 
cultural  terrain  and  using  their  assessments  to  successfully  control  the  course  of  all  their  cross- 
cultural  encounters.  They  have  a  clear  awareness  of  their  own  cultural  biases  as  well  as  a  strong 
ability  to  predict  both  the  short-  and  long-term  effects  of  their  cultural  interactions  on  mission 
objectives. 
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Next,  a  chart  provides  information  to  the  Soldier  about  how  he  or  she  scored  on  each  of 
the  five  factors  (see  Figure  E-5  in  Appendix  E).  Following  this  is  an  extended  description  of 
each  factor  including  what  it  is,  why  it  is  important,  what  it  looks  like  at  different  performance 
levels,  and  how  to  improve  it.  As  an  example,  the  feedback  provided  for  Cultural  Acuity  is 
shown  in  Appendix  F. 

Limitations  and  Future  Research 

This  effort  focused  on  developing  and  refining  a  3C  assessment  battery  based  on  a 
developmental  model  of  3C.  Although  a  valid  and  reliable  package  of  measures  as  supported  by 
statistical  findings  was  developed,  there  would  be  significant  benefit  to  extending  the  effort  to  a 
broad  range  of  members  from  the  potential  user  community  in  the  Anny  to  overcome  some 
limitations.  The  current  study  was  constrained  in  that  the  researchers  relied  on  convenience 
samples  of  Soldiers  which  included  both  Officers  and  Enlisted  as  well  as  ROTC  cadets.  Future 
research  should  detennine  whether  the  findings  and  related  measures  can  generalize  across 
grade,  mission  set,  and  Military  Occupational  Specialty.  Additionally,  as  previously  mentioned, 
responses  to  supervisor  and  peer  ratings  were  only  administered  to  cadets’  supervisors  and  peers, 
and  then  only  following  a  two-week  immersion  experience.  This  participant  group  and  their 
related  overseas  experiences  are  dramatically  different  than  Soldiers  and  deployments.  It  would 
be  unwise  to  assume  that  1)  cadets  can  have  significant,  observable,  and  lasting  gains  in  3C  from 
such  a  short  experience,  and  2)  supervisors  and  peers  would  be  in  a  position  to  observe  and 
accurately  assess  such  gains  if  they  did  occur.  These  limitations  prevent  us  from  making  strong 
generalizations  from  this  dataset  to  a  Soldier  population  and  from  establishing  criterion  validity 

Though  there  were  significant  correlations  between  the  five  factors  as  measured  by  the 
report  and  individual  SJT  and  vignette  questions,  the  size  of  the  correlations  were  small. 
Additionally,  numerous  SJT  and  vignette  questions  either  did  not  correlate  with  any  of  the  five 
factors  or  were  significantly  correlated  with  multi  factors.  Despite  these  issues,  each  vignette 
and  SJT  question  was  associated  with  a  single  factor  and  contributes  to  an  individual’s  overall 
score  on  that  factor  as  a  way  to  include  both  scenario  based  and  self-report  measures  into  the 
final  battery.  As  both  self-report  items  and  scenario-based  questions  were  significantly 
correlated  with  peer  and  supervisor  ratings,  future  research  should  attempt  to  develop  an 
empirically  sounder  approach  to  incorporating  the  scenario-based  measures  with  the  self  report 
measure  to  calculate  an  overall  score.  One  approach  would  be  to  conduct  a  regression  analysis 
using  the  five  factors,  the  overall  SJT  score,  and  the  overall  vignette  score  to  determine  if  each 
component  accounts  for  a  significant  portion  of  the  variance  in  a  criterion  measure  such  as  peer 
or  supervisor  ratings. 

Another  limitation  associated  with  combining  the  scores  on  the  self-report,  SJT,  and 
vignettes  into  a  single  score  is  the  use  of  relative  scoring  as  opposed  to  absolute  scoring.  To 
calculate  a  combined  score,  a  participant’s  score  on  each  of  the  three  types  of  measures  is 
standardized  by  computing  a  Z-score,  thus  ensuring  that  a  participant’s  performance  is  evaluated 
as  compared  to  other  participants  as  opposed  to  an  objective  standard.  Future  research  should 
modify  the  scoring  algorithm  to  use  an  absolute  scoring  system. 
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Finally,  the  tenn  Cultural  Acuity  may  not  be  the  best  label  for  the  third  factor  in  our 
model.  The  majority  of  these  questions  are  reversed  scored  and  ask  the  participant  whether  or 
not  they  would  have  trouble  performing  a  specific  task  such  as  predicting  the  long  term  effects  of 
their  actions  or  understanding  the  intent  of  a  foreign  national.  Thus,  a  term  such  as  Self  Efficacy 
may  prove  to  be  a  better  name  for  this  factor. 

Future  research  efforts  should  focus  on  establishing  the  reliability  and  validity  of  the 
finalized  battery.  The  research  team  intends  to  administer  the  assessment  battery  to  a  new 
rotation  of  20  CA  Soldiers  in  three  phases:  in  the  first  phase,  new  Soldiers  have  already  taken  the 
battery  in  the  early  stages  of  their  CA  pre-deployment  training.  In  the  second  phase,  the  battery 
will  be  administered  again  to  the  Soldiers  following  pre-deployment  training.  The  CA  staff  who 
trained  the  Soldiers  for  their  deployments  will  also  complete  a  series  of  criterion  measures.  In 
the  third  stage,  the  battery  will  be  re-administered  to  the  20  Soldiers  (including  team  leaders) 
shortly  after  they  return  from  their  nine-month  deployments.  This  third  assessment  package  will 
include  supervisor  and  peer  rating  criterion  measures.  Responses  will  be  based  on  a  nine-month 
period  of  observations  and  interactions  with  fellow  Soldiers  and  subordinates  during  the 
deployment  and  will  provide  an  objective  measure  of  field  perfonnance.  The  data  will  be 
assessed  from  the  testing  phases  by  establishing  test-retest  reliability,  construct  validity,  and 
finally,  criterion  validity  based  on  supervisor  and  peer  ratings.  If  needed,  the  battery  will  be 
refined  further  to  address  the  findings.  In  addition,  the  future  three-phase  study  will  allow  the 
researchers  to  examine  the  improvements  in  3C  as  a  function  of  Soldiers’  training  and 
deployment  using  the  battery.  In  general,  the  assessment  battery  will  benefit  from  data  collected 
from  a  wide  range  of  demographics  of  Soldiers  as  they  undergo  training  programs  and  actual 
field  experience. 

The  researchers  also  hope  to  further  develop  and  administer  our  measures  with  additional 
Army  audiences,  possibly  including  multiple  TRADOC  installations.  Additional  areas  for 
further  development  include  providing  assessment  feedback  at  the  unit  level  to  leaders  and 
developing  and  validating  assessment  content  that  would  be  directly  relevant  to  intelligence 
missions  and/or  other  levels  of  operations. 

The  researchers  also  believe  that  there  could  be  a  benefit  to  studying  perspective  taking 
ability.  Although  significant  research  has  been  conducted  on  perspective  taking  in  monocultural 
environments,  few  research  efforts  have  focused  on  multicultural  perspective  taking,  and  in 
particular,  with  Soldiers  operating  in  multicultural  settings.  By  incorporating  multiple 
methodologies  and  leveraging  empirical  data  from  this,  and  other  efforts,  we  believe  there  would 
be  value  in  creating  a  model  of  the  dispositional  and  environmental  factors  that  potentially 
impact  multicultural  perspective  taking  in  Soldiers  that  would  apply  across  multiple  mission  sets. 
This  model  could  serve  as  the  basis  for  an  instrument  that  assesses  multicultural  perspective 
taking. 
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Appendix  A 


Final  3C  Assessment  Battery 


A-l 


SUBJECT  NAME: 


(Lname,  Fname) 


CROSS-CULTURAL 

ASSESSMENT 

MEASURE 

ARMY  RESEARCH  INSTITUTE 
361  INTERACTIVE,  LLC 


This  Exercise  consists  of  3  parts.  Please  read  the 
directions  for  each  part  carefully  and  answer  each 
question  as  accurately  as  possible.  The  entire  measure 
should  take  no  longer  than  30  minutes  to  complete. 
Thank  you  very  much  for  your  honesty  and  attention  to 

detail  in  your  responses. 
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PART  1  OF  3 


INSTRUCTIONS :  FOR  EACH  ITEM,  PLEASE  CIRCLE  THE  NUMBER  THAT  BEST 
REFLECTS  YOUR  LEVEL  OF  AGREEMENT  WITH  THAT  STATEMENT. 

6  =  Strongly  Agree 

Strongly  Agree 

5  =  Moderately  Agree 

Moderately  Agree 

4  =  Slightly  Agree 

3  =  Slightly  Disagree 

2  =  Moderately  Disagree 

1  =  Strongly  Disagree 

Slightly  Agree 

Slightly  Disagree 

Moderately  Disagree 

Strongly  Disagree 

1. 

I  would  have  trouble  predicting  the  long-term  effects  of  my  actions  in  a  new  country . 

1 

2 

3 

4 

5 

6 

2. 

1  would  easily  change  my  outward  appearance  based  on  the  mission,  such  as  switching  from 
a  military  to  a  humanitarian  effort . 

1 

2 

3 

4 

5 

6 

3. 

On  a  deployment,  I  would  be  good  at  “working  the  locals”  to  give  me  needed  intel . 

1 

2 

3 

4 

5 

6 

4. 

The  views  and  beliefs  of  American  culture  are  generally  superior  to  those  of  the  countrie(s) 
we  visit . 

5. 

My  personality  is  such  that  most  people  are  quickly  drawn  to  me. . . . 

1 

2 

3 

4 

5 

6 

6. 

I  often  have  trouble  envisioning  the  long-term  effects  of  my  actions. 

1 

2 

3 

4 

5 

6 

7. 

1  am  good  at  getting  others  to  see  my  point  of  view . 

1 

2 

3 

4 

5 

6 

8. 

1  do  better  sticking  with  an  approach  until  it  works  versus  changing  tactics . 

1 

2 

3 

4 

5 

6 

9. 

I  would  befriend  locals  during  deployments  to  support  mission  success . 

1 

2 

3 

4 

5 

6 

10. 

I  often  have  to  rely  on  others  to  adjust  my  perceptions  of  what  is  really  going  on  in  a  group 
setting . 

1 

2 

3 

4 

5 

6 

11. 

As  an  American,  I  probably  do  not  have  as  many  biases  as  do  people  from  Middle  Eastern 
cultures . 

1 

2 

3 

4 

5 

6 

12. 

Without  the  help  of  fellow  Soldiers,  I  would  struggle  in  figuring  out  what  the  locals  are 
really  up  to  in  deployment  situations . 

1 

2 

3 

4 

5 

6 

13. 

1  would  quickly  get  used  to  unfamiliar  customs  if  deployed . 

1 

2 

3 

4 

5 

6 

14. 

1  devote  significant  time  to  building  many  lasting  relationships  in  my  life . 

1 

2 

3 

4 

5 

6 

15. 

I  often  “feel  the  pain”  of  others  when  someone  is  sharing  a  sad  story 

1 

2 

3 

4 

5 

6 

16. 

If  1  knew  I  was  being  deployed,  1  would  spend  some  free  time  learning  about  the  cultural 
customs  before  1  left . 

1 

2 

3 

4 

5 

6 

17. 

I  would  easily  and  believably  “fake  compassion”  with  foreign  citizens  to  achieve  the 
mission . 

1 

2 

3 

4 

5 

6 
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6  =  Strongly  Agree 

Strongly  Agree 

5  =  Moderately  Agree 

Moderately  Agree 

4  =  Slightly  Agree 

3  =  Slightly  Disagree 

2  =  Moderately  Disagree 

1  =  Strongly  Disagree 

Slightly  Agree 

Slightly  Disagree 

Moderately  Disagree 

Strongly  Disagree 

18. 

1  find  the  thought  of  negotiating  with  foreign  village  elders  unpleasant . 

1 

2 

3 

4 

5 

6 

19. 

My  own  sense  of  humor  would  come  in  handy  during  deployments  to  put  foreign  locals  at 
ease . 

1 

2 

3 

4 

5 

6 

20. 

Deployed  US  forces  need  to  focus  less  on  compassion  and  more  on  ‘ 
when  dealing  with  locals . 

‘getting  the  job  done” 

1 

2 

3 

4 

5 

6 

21. 

I  would  find  it  easy  to  be  casual  and  friendly  with  foreign  citizens  during  deployments . 

1 

2 

3 

4 

5 

6 

22. 

When  watching  two  people  have  a  discussion,  I  can  pick  up  on  any  differences  between 
what  is  being  said  and  what  is  really  felt . 

1 

2 

3 

4 

5 

6 

23. 

1  enjoy  making  sense  of  complex  situations . 

1 

2 

3 

4 

5 

6 

24. 

Interacting  with  locals  in  order  to  build  relationships  during  deployments  would  be  worth  the 
risks . 

1 

2 

3 

4 

5 

6 

25. 

I  sometimes  wonder  how  my  own  culture  influences  how  I  see  things . 

1 

2 

3 

4 

5 

6 

26. 

My  personality  is  such  that,  in  a  foreign  country,  I  could  quickly  put  an  irate  citizen  at 
ease . 

1 

2 

3 

4 

5 

6 

27. 

1  possess  the  skills  needed  to  persuade  foreign  civilians  to  provide  sensitive  information. 

1 

2 

3 

4 

5 

6 

28. 

I  consider  myself  as  being  oblivious  to  what  is  really  going  on  in  group  interactions . 

1 

2 

3 

4 

5 

6 

29. 

I  would  have  little  problem  figuring  out  the  heart  of  the  matter  when  observing  a 
disagreement  between  Soldiers  and  foreign  citizens . 

1 

2 

3 

4 

5 

6 

30. 

It  is  easy  for  me  to  quickly  gain  the  trust  of  others  through  casual  discussion . 

1 

2 

3 

4 

5 

6 

31. 

if  1  find  a  common  practice  of  the  locals  offensive  while  deployed,  1  would  have  trouble 
understanding  why  the  locals  act  that  way . 

1 

2 

3 

4 

5 

6 

32. 

Prior  to  a  deployment,  I  would  try  to  learn  the  basics  of  the  language  before  going,  whether 
directed  to  or  not . 

1 

2 

3 

4 

5 

6 

33. 

Since  we  are  often  deployed  in  order  to  help  other  countries,  those  countries  should  adjust  to 
our  customs,  not  the  other  way  around . 

1 

2 

3 

4 

5 

6 

34. 

I  can  win  over  a  group  of  strangers  with  ease . 

1 

2 

3 

4 

5 

6 

35. 

I  would  probably  rely  on  another  team  member  to  strike  up  initial  conversations  with  foreign 
citizens  when  deployed,  as  this  is  not  my  strong  suit . 

1 

2 

3 

4 

5 

6 
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6  =  Strongly  Agree 

Strongly  Agree 

5  =  Moderately  Agree 

Moderately  Agree 

4  =  Slightly  Agree 

3  =  Slightly  Disagree 

2  =  Moderately  Disagree 

1  =  Strongly  Disagree 

Slightly  Agree 

Slightly  Disagree 

Moderately  Disagree 

36. 

I  could  see  my  temper  getting  the  best  of  me  when  interacting  with  i 
citizens  during  a  deployment . 

Strongly  Disagr 

nappreciative  foreign 

ee 

1 

2 

3 

4 

5 

6 

37. 

I  am  a  compassionate  and  trusting  person  in  general . 

1 

2 

3 

4 

5 

6 

38. 

It  would  be  hard  for  me  to  read  the  intent  of  a  foreign  citizen  with  whom  1  am 
communicating . 

1 

2 

3 

4 

5 

6 

39. 

1  use  my  sense  of  humor  to  quickly  put  people  at  ease . 

1 

2 

3 

4 

5 

6 

40. 

If  a  trainee  was  resistant  to  my  instructions,  1  would  put  myself  in  their  shoes  to  figure  out 
why . 

1 

2 

3 

4 

5 

6 

41. 

In  trying  to  persuade  a  village  elder  to  let  us  search  his  village,  1  would  probably  fall  back  on 
force  if  my  first  attempts  at  persuasion  did  not  work . 

1 

2 

3 

4 

5 

6 

42. 

If  you  know  the  basic  do’s  and  don’ts  of  a  country,  and  some  language,  that’s  all  you  need  to 
get  by  to  interact  with  locals  during  deployments . 

1 

2 

3 

4 

5 

6 

43. 

Negotiating  with  village  leaders  during  a  deployment  would  fit  my  abilities . 

1 

2 

3 

4 

5 

6 

44. 

I  get  upset  when  1  hear  people  making  fun  of  people  from  other  countries . 

1 

2 

3 

4 

5 

6 
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Appendix  B 


Peer  and  Supervisor  Rating/Tanking  Scale 
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POST¬ 

DEPLOYMENT 

SUPERVISOR 

MEASURE 

ARMY  RESEARCH  INSTITUTE 

Please  read  the  directions  carefully  and  answer  each 
question  as  accurately  as  possible.  Thank  you  for  your 
attention  to  detail  in  your  responses. 
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CROSS-CULTURAL  SUPERVISOR  RATINGS  AND  RANKINGS:  INSTRUCTIONS 


IMPORTANT:  PLEASE  PAY  CLOSE  ATTENTION  TO  THE  INSTRUCTIONS.  YOUR 
OPINIONS  ARE  INVALUABLE  TO  OUR  RESEARCH.  THANK  YOU! 

The  following  questions  address  the  perfonnance  effectiveness  of  EACH  OF  THE  CADETS  IN 
YOUR  UNIT  in  a  cross-cultural  setting.  Answering  each  question  is  a  TWO-STEP  process  that 
asks  you  to  both  RATE  and  RANK  the  cadets  in  your  unit.  Please  read  the  following  example 
demonstrating  this  process  before  you  proceed. 


Example: 

Imagine  that  your  unit  is  made  up  of  twelve  cadets:  1)  W.  James  2)  J.  Dewey  3)  C.  Peirce  4)  J. 
Mill  5)  U.  Grant  6)  L.  Grossman  7)  D.  Mitchell  8)  J.  Diamond  9)  P.  Wylie  10)  J.  Aaron  11  )J. 
Stewart  12)  F.  Bacon 

Step  One:  RATING 

Please  provide  a  rating  for  EACH  CADET  IN  YOUR  UNIT  by  placing  his/her  name  in  the 
category  that  best  represents  his/her  level  of  performance. 

•  A  category  can  have  multiple  individuals. 

•  Please  try  to  use  all  categories. 

•  Each  person  should  be  placed  into  a  single  category.  Do  not  put  an  individual  into 
multiple  categories. 

•  Please  use  each  cadet’ s  FIRST  INITIAL  and  LAST  NAME. 


Demonstrates  no  knowledge 
of  a  foreign  language 

Demonstrates  limited 
knowledge  of  a  foreign 
language;  Only  knows 
greetings  and  courtesies 

Demonstrates  sufficient 
knowledge  of  a  foreign 
language  to  get  by  on  a  daily 
basis 

Demonstrates  fluent 
knowledge  of  a  foreign 
language 

U.  Grant 

P.  Wylie 

J.  Aaron 

J.  Dewey 

C.  Peirce 

J.  Stewart 

F.  Bacon 

W.  James 

J.  Mill 

D.  Mitchell 

L.  Grossman 

J.  Diamond 

Step  Two:  RANKING 

Within  each  category,  RANK  the  individuals  based  on  their  level  of  effectiveness  compared  to 
each  other  ( 1  being  the  most  effective).  For  example,  if  you  place  4  people  in  the  same  category, 
rank  them  1-4  by  placing  the  number  next  to  his/her  name. _ 


Demonstrates  no  knowledge 
of  a  foreign  language 

Demonstrates  limited 
knowledge  of  a  foreign 
language;  Only  knows 
greetings  and  courtesies 

Demonstrates  sufficient 
knowledge  of  a  foreign 
language  to  get  by  on  a  daily 
basis 

Demonstrates  fluent 
knowledge  of  a  foreign 
language 

U.  Grant  2 

C.  Peirce  1 

P.  Wylie  1 

J.  Stewart  2 

J.  Aaron  4 

F.  Bacon  2 

J.  Mill  J 

L.  Grossman  1 

J.  Dewey  2 

W.  James  1 

D.  Mitchell  J 

J.  Diamond  4 

Thus,  U.  Grant  and  C.  Peirce  are  the  least  effective  people  (but  C.  Peirce  is  better  than  U.  Grant), 
and  J.  Dewey,  W.  James,  D.  Mitchell,  and  J.  Diamond  are  the  best  (but  W.  James  is  better  than  J. 
Dewey  who  is  better  than  D.  Mitchell,  who  is  better  than  J.  Diamond). 

Please  continue  to  the  next  page  to  rate/rank  your  unit 

B-3 


Cross-Cultural  Supervisor  Ratings  and  Rankings:  Questions 


Please  list  all  of  the  cadets  in  your  unit  using  his/her  First  Initial  and  Last  Name: 

1.  _ 

2.  _ 

3.  _ 

4.  _ 

5.  _ 

6.  _ 

7.  _ 

8.  _ 

9.  _ 

10.  _ 

11. _ 

12. _ 

13.  _ 

14.  _ 

15.  _ 

16.  _ 

17.  _ 

18.  _ 

19.  _ 

20.  _ 

21.  _ 

22.  _ 

23.  _ 

24. 
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Please  rank  and  rate  all  the  above-mentioned  cadets  based  on  the  following  behaviors: 


1.  Culture  Fundamentals 

How  effective  are  your  cadets  at  demonstrating  knowledge  of  culture  fundamentals  such  as 
definitions  of  culture,  values,  beliefs,  behaviors,  and  norms? 


Does  not  apply  customs  and 
courtesies  outside  of  own 
culture 

Applies  limited  relevant 
aspects  of  culture;  Equates 
cultural  differences  purely  to 
customs  and  taboos 

Demonstrates  sufficient 
application  and  knowledge  of 
cross-cultural  values,  beliefs, 
behaviors,  and  norms 

Demonstrates  superior  cross- 
cultural  competence; 
Demonstrates  characteristics 
that  enable  learning  and 
adaptations  to  unfamiliar 
cultures 

2.  Culture  Awareness 

How  effective  are  your  cadets  at  demonstrating  awareness  of  cross-cultural  differences? 


Demonstrates  no  awareness 
of  American,  Army,  and 
other  cultures  in  regards  to 
religion,  ethnicity, 
sex/gender,  social  class,  or 
regional  differences 


Demonstrates  limited 
awareness  and  understanding 
of  American  cultures,  Army 
culture,  and  other  cultures 


Demonstrates  sufficient 
awareness  that  cultural 
differences  play  a  role  in 
cross-  cultural  interactions 


Demonstrates  superior 
awareness  of  own  biases  and 
does  not  allow  them  to 
influence  own  perceptions 
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3.  Culture  Skills 

How  effective  are  your  cadets  at  applying  cross-cultural  skills  in  rapport  building  with  members 
of  other  cultures? 


Does  not  demonstrate 
rapport-building  skills; 
Avoids  engagement  with 
members  of  other  cultures 

Makes  limited  attempts  at 
verbal  and  non-verbal 
communication  with 
members  of  other  cultures; 
Has  difficulty  in  considering 
other's  perspectives;  Has 
difficulty  in  suspending 
judgment 

Performs  sufficiently  at 
verbal  and  non-verbal 
communication  as  part  of 
rapport-building 

Performs  successfully  at 
rapport-building,  considering 
other's  perspectives,  and 
suspending  judgment 

4.  Communication  Skills 

How  effective  are  your  cadets  at  communicating  with  members  of  other  cultures? 


Does  not  communicate 
verbally  or  non-verbally  with 
members  of  other  cultures; 

Does  not  develop 
relationships  with  members 
of  other  cultures 


Demonstrates  limited  verbal 
and  non-verbal 
communication  techniques, 
but  application  of  techniques 
is  not  effective 


Applies  sufficiently 
appropriate  verbal  and  non¬ 
verbal  communication 
techniques  to  most 
communications 


Demonstrates  superior 
performance  in 
communicating  with 
members  of  other  cultures  by 
speaking,  gesturing,  and 
listening 
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5.  Cross-Cultural  Operations 

How  effective  are  your  cadets  at  integrating  cultural  considerations  into  developing  and 
executing  plans? 


Does  not  integrate  cultural 
considerations  when 
developing  and  executing 
plans 

Integrates  limited  cultural 
considerations  when 
developing  and  executing 
plans,  but  does  not  consider 
their  importance 

Sufficiently  integrates 
cultural  considerations  when 
developing  and  executing 
plans 

Successfully  balances 
planning  and  cultural 
considerations  to  reduce 
effects  of  negative 
consequences  without 
compromising  the  mission 

6.  Influence 

How  effective  are  your  cadets  at  practicing  negotiation,  persuasion,  and  mediation  in  a  cross- 
cultural  setting? 


Is  unable  to  exert  influence 
over  members  of  other 
cultures 

Makes  limited  attempts  to 
negotiate  with  members  of 
other  cultures;  Demonstrates 
limited  consideration  of 
social  and  political  position 
of  members  of  other  cultures 

Sufficiently  applies  a  number 
of  negotiation  and  mediation 
techniques 

Successfully  balances 
seamless  and  efficient 
negotiation/mediation  with 
mission  goals 

B-7 


B-8 


Appendix  C 


Participating  Soldiers’  Branch  and  MOS 
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Table  C-l 


Participating  Soldiers’  branch  and  MOS 


Branch  and  MOS 

Count 

% 

Annor 

19A 

44 

16.30% 

19C 

3 

1.11% 

19D 

7 

2.59% 

19K 

7 

2.59% 

19Z 

1 

0.37% 

Unknown 

10 

3.70% 

Total 

72 

26.67% 

Civil  Affairs 

38A 

23 

8.52% 

38B 

27 

10.00% 

Unknown 

1 

0.37% 

Total 

51 

18.89% 

Military  Intelligence 

35A 

1 

0.37% 

35D 

32 

11.85% 

35E 

1 

0.37% 

35F 

1 

0.37% 

35P 

1 

0.37% 

Unknown 

2 

0.74% 

Total 

38 

14.07% 

Military  Police 

31B 

30 

11.11% 

31A 

1 

0.37% 

Total 

31 

11.48% 

Infantry 

11A 

3 

1.11% 

1  IB 

24 

8.89% 

nc  _ 

3 

1.11% 

Total 

30 

11.11% 

Medical  CMF 

68W 

9 

3.33% 

68X  _ 

1 

0.37% 

Total 

10 

3.70% 

Aviation 

15A 

5 

1.85% 

15B 

1 

0.37% 

Unknown 

1 

0.37% 

Total 

7 

2.59% 
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Table  C-l  (continued) 


Air  Defense  Artillery 


14A 

4 

1.48% 

14Z 

1 

0.37% 

Total 

5 

1.85% 

Mechanical  Maintenance  CMF 

91B 

1 

0.37% 

91K 

1 

0.37% 

91X 

1 

0.37% 

91A 

1 

0.37% 

Total 

4 

1.48% 

Adjutant  General/ Army  Band 

42A 

2 

0.74% 

42H 

1 

0.37% 

Total 

3 

1.11% 

Corps  of  Engineers 

12Y 

1 

0.37% 

21A 

1 

0.37% 

21Y 

1 

0.37% 

Total 

3 

1.11% 

Logistics 

90A  _ 

3 

1.11% 

Total 

3 

1.11% 

Signal  Corps 

25A 

2 

0.74% 

25W  _ 

1 

0.37% 

Total 

3 

1.11% 

Electronic  Maintenance  &  Calibrations  CMF 

94W 

1 

0.37% 

Total 

1 

0.37% 

Field  Artillery 

13A 

1 

0.37% 

Total 

1 

0.37% 

Health  Services 

70B  _ 

1 

0.37% 

Total 

1 

0.37% 
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Table  C-l  (continued) 


Immaterial  &  Personnel  Special  Reporting 

Codes 

03A  _ 

1 

0.37% 

Total 

1 

0.37% 

Infonnation  Operations 

30A  _ 

1 

0.37% 

Total 

1 

0.37% 

Medical  Service  Corps 

67J  _ 

1 

0.37% 

Total 

1 

0.37% 

Preventive  Medicine  Sciences 

Unknown 

1 

0.37% 

Total 

1 

0.37% 

Psychological  Operations 

37X  _ 

1 

0.37% 

Total 

1 

0.37% 

Quartermaster  Corps 

92Y 

1 

0.37% 

Total 

1 

0.37% 

Public  Affairs 

46A 

1 

0.37% 

Total 

1 

0.37% 

270 

100% 

A  =267 

Note:  2  participants  belong  to  > 

1  Branch/MOS 
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Cadet  Cultural  Immersion  Training  Impact 
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Cadet  Cultural  Immersion  Training  Impact 


The  findings  described  below  highlight  the  benefits  of  a  two-week,  overseas  immersion 
experience  in  a  number  of  locations  for  ROTC  cadets.  Results  are  based  on  pre-  and  post- test 
questionnaire  responses  collected  from  ROTC  cadets  who  typically  operated  in  squad-  to 
platoon-sized  units  while  abroad.  Participants  performed  significantly  better  on  the  post-test  self 
report  compared  to  the  pre-test  self-report;  however  there  was  no  difference  between  their 
performance  on  the  post- test  SJTs  and  vignettes  compared  to  the  pre-test  SJTs  and  vignettes.  We 
believe  that  lack  of  significant  differences  between  pre-test  and  post-test  SJTs  and  vignettes,  as 
compared  to  a  significant  difference  between  pre-test  and  post-test  self-report,  may  be  explained 
by  the  fact  that  a  two  week  overseas  immersion  experience  may  not  be  enough  time  to  impact  a 
person’s  ability  to  perform  better  on  a  complex  and  multidimensional  measure  such  as  an  SJT. 

A  self-report  measure,  on  the  other  hand,  can  be  influenced  by  such  an  experience,  as 
participants  may  have  felt  more  confident  in  their  abilities  after  their  immersion  experience  and 
thus  assessed  their  own  capabilities  more  favorably  than  pre-deployment.  Alternatively,  the 
SJTs  and  vignettes  may  not  have  been  sensitive  enough  measures  to  detect  any  changes  a  two- 
week  immersion  experience  may  have  caused. 

Self  Report  Results  and  Discussion 

Cadets’  responses  reflected  an  increase  in  cross-cultural  competence  as  a  function  of  the 
immersion  experiences.  We  used  paired-samples  /-tests  to  compare  their  pre-  and  post¬ 
deployment  responses.  First,  we  compared  the  means  of  all  the  pre-test  and  all  the  post- test 
items.  A  significant  difference  was  found.  Specifically,  cadets  scored  higher  on  the  post-test  than 
they  did  on  the  pre-test,  t(  1 1 9)  =  -4.74 ,p  <  .01.  Cadets  provided  responses  that  were  significantly 
higher  on  the  post-test  following  the  cultural  immersion  program.  In  addition,  of  our  five 
theorized  components  of  cross-cultural  competence  (3C),  Cultural  Acuity,  Cultural  Maturity,  and 
Interpersonal  Skills  showed  significant  increases  from  pre-test  to  post-test 

Cultural  Acuity 

Cultural  Acuity  is  the  ability  to  form  accurate  cross-cultural  understandings  and 
assessments  of  situational  dynamics,  the  perspectives  of  others,  and  the  impact  of  cultural  actions 
on  the  broader  mission.  Items  that  appeared  on  the  Cultural  Acuity  scale  address  cadets’  abilities 
to  predict  unfolding  events,  effectiveness  of  prioritizing  actions,  and  determining  reasons  for  an 
observed  argument.  A  paired-samples  /-test  showed  a  significant  difference  between  pre-test  and 
post-test  responses  on  Cultural  Acuity.  Cadets  across  all  immersion  experiences  showed  a 
significant  increase  in  Cultural  Acuity  from  pre-test  to  post-test  (7(1 19)  =  -0.55,  p  <  ■01), 
suggesting  an  improvement  in  their  ability  to  assess  situations  in  cross-cultural  settings.  Analysis 
of  individual  components  of  Cultural  Acuity  showed  that  cadets  reported  significant  gains  in 
Perspective  Taking,  Integration/Big  Picture  Assessment  Ability  and  Sensemaking.  Overall,  the 
wide  range  of  activities  that  cadets  experienced  during  their  deployment  significantly  contributed 
to  their  understanding  of  what  to  pay  attention  to  in  various  situations,  around  people,  and  in  the 
context  of  goals  to  be  achieved. 
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Cultural  Maturity 


Cultural  Maturity  is  the  ability  to  remain  confident,  calm,  and  dedicated  in  cross-cultural 
settings,  and  to  seek  interactions  to  promote  mission  success.  Items  that  appeared  on  this  scale 
address  one’s  enjoyment  level  of  meeting  different  people  and  the  likelihood  of  learning  the 
basics  of  the  relevant  language  prior  to  deployment.  Cross-cultural  immersion  exposed  cadets  to 
a  variety  of  situations  within  a  different  culture.  These  experiences  contributed  to  cadets’ 
confidence  to  operate  effectively  in  cross-cultural  settings,  as  reflected  by  their  responses.  The 
Cultural  Maturity  component  showed  a  significant  increase  from  pre-test  to  post-test,  cadets 
scored  higher  on  the  Cultural  Maturity  items  on  the  post-test  than  they  did  on  the  pre-test  (?(1 19) 
=  -2.26,  p  <  .05).  Within  Cultural  Maturity,  cadets  reported  an  increased  Willingness  to  Engage 
with  local  populaces  after  their  overseas  experiences.  Overall,  findings  suggest  that  these  cadets 
are  more  likely  to  seek  out  cross-cultural  situations  and  are  more  confident  operating  within 
them,  as  a  function  of  the  cultural  immersion  program. 

Interpersonal  Skills 

Interpersonal  Skills  is  the  ability  to  consistently  present  oneself  in  a  manner  that 
promotes  positive  short-  and  long-term  relationships  in  order  to  achieve  mission  objectives. 

Items  that  that  appeared  on  Interpersonal  Skills  scale  address  one’s  ability  to  get  others  to  see 
one’s  point  of  view,  put  upset  people  at  ease,  and  gain  trust  of  others.  By  interacting  with  foreign 
nationals,  cadets  got  a  sense  of  how  and  what  they  think,  thereby  developing  trust  and 
understanding.  We  found  a  significant  difference  between  pre-test  and  post-test,  suggesting  an 
increase  in  competency  as  a  result  of  the  immersion  program,  cadets  scored  higher  on  the  post¬ 
test  than  they  did  on  the  pre-test  (t(l  19)  =  -6.88, p  <  .01).  Within  Interpersonal  Skills,  cadets 
reported  gains  in  three  different  areas.  Significant  gains  were  reported  for  Relationship  Building 
ability,  Rapport  Building  ability  and  Manipulation/Persuasion  ability.  Because  the  immersion 
program  gave  cadets  numerous  opportunities  to  interact  with  foreign  citizens  ranging  from 
sharing  meals  to  collaborating  in  completing  tasks,  cadets  are  now  more  comfortable  interacting 
with  people  from  foreign  cultures. 

Conclusion 

The  outcome  of  the  two-week  ROTC  cultural  immersion  program  is  a  group  of  cadets 
that  possess  a  great  deal  more  confidence  in  their  3C  than  they  started  with,  as  supported  by  their 
responses.  The  significant  gains  on  a  number  of  cross-cultural  components  and  KSAAs  provide 
support  for  the  benefits  of  the  program.  Through  a  cross-cultural  assessment  battery,  we  captured 
the  span  and  the  depth  of  the  experiences  that  cadets  had.  They  had  numerous  interactions  with 
foreign  nationals  ranging  from  friendly  conversation,  collaboration,  competition,  and  instruction, 
while  operating  in  a  culture  different  from  their  own.  The  interactions  contributed  to  cadets’ 
awareness  of  the  differences  and  the  similarities  between  themselves  and  foreigners.  The 
discovered  significant  gains  from  pre-deployment  to  post-deployment  responses  show  that 
cadets’  future  actions  on  foreign  soil  are  more  likely  to  reflect  such  awareness  in  taking  on  both 
challenges  and  opportunities.  Although  our  sample  was  comprised  of  cadets  with  a  range  of  prior 
cultural  experiences  who  benefited  from  the  program,  such  an  immersion  program  may  be  of 
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particular  benefit  to  cadets  lacking  prior  cross-cultural  experience.  For  scenario-based  measures, 
analyses  did  not  reveal  statistically  significant  gains  in  perfonnance,  suggesting  that  a  two-week 
experience  is  not  sufficient  to  generate  a  difference  in  such  multi-faceted  measures  that  represent 
real-world  complexity.  Although,  analyses  revealed  convergent  validity  between  self-report  and 
scenario-based  measures,  we  found  that  the  latter  have  more  face  validity. 
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Appendix  E 


Computer  Based  Assessment  Battery  Screenshots 
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Cross  Cultural  Assessment  Battery 


The  following  questions  and  scenarios  are  intended  to  give  you  a  sense  of  the  individual 
strengths  and  challenge  areas  that  you,  as  a  Soldier,  may  have  when  interacting  with 
people  from  other  cultures  while  performing  your  mission. 

To  ensure  that  you  get  the  most  helpful  and  accurate  scoring  and  feedback,  it  is  important 
that  you  answer  openly  and  honestly.  Following  the  questions  (which  should  take  no  more 
than  30  minutes  to  answer),  you  will  receive  a  series  of  scores  along  with  guidance  on 
how  to  interpret  the  scores  and  use  the  feedback  provided.  The  goal  of  this  tool  is  to 
provide  you  with  feedback  on  how  to  leverage  your  cultural  strengths  and  manage  any 
weaknesses  to  better  achieve  your  mission. 


Click  Continue  when  you  are  ready  to  begin.  You  will  not  be  timed. 


(  Continue  ) 


Figure  E-l.  Instruction  screen. 


Part  1  b 


c 


Deployed  US  forces  need  to  focus  less  on  compassion  and  more  on  “getting  the  job 
done*'  when  dealing  with  locals. 

I  often  have  to  rely  on  others  to  adjust  my  perceptions  of  what  is  really  going  on  in 
a  group  setting. 

I  am  a  compassionate  and  trusting  person  in  general. 

I  use  my  sense  of  humor  to  quickly  put  people  at  ease. 

If  I  find  a  common  practice  of  the  locals  offensive  while  deployed,  I  would  have 
trouble  understanding  why  the  locals  act  that  way. 

I  often  have  trouble  envisioning  the  long-term  effects  of  my  actions. 

My  personality  is  such  that  most  people  are  quickly  drawn  to  me. 

I  devote  significant  time  to  building  many  lasting  relationships  in  my  life. 
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(  Continue  ) 


Figure  E-2.  Self  Report  questions. 
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Situation  1:  The  New  Interpreter 

You  are  a  platoon  leader  who  has  recently  been  deployed  on  a  six  month  tour  to  Honduras  for  a  humanitarian  mission.  The  emphasis  of  this  deployment  is  on 
bringing  a  better  quality  of  life  to  the  people  of  Honduras  through  food  distribution  and  MEDCAPs. 

Although  three  members  of  your  platoon  can  speak  very  limited  Spanish,  you  have  been  assigned  a  new  interpreter  by  HHQ.  Other  than  the  fact  that  HHQ  has 
assigned  this  interpreter,  you  know  nothing  about  this  interpreter,  his  qualifications  or  his  objectives. 


When  you  and  your  platoon  land  at  an  airport  in  a  remote  region  of  Honduras,  you  are  surprised  to  find  that  your  interpreter  is  not  there  to  meet  you  as  planned.  He 
was  to  direct  you  on  where  to  go  next  to  meet  with  local  leaders  to  discuss  the  food  distribution  program  that  you  hope  to  coordinate  in  multiple  area  villages.  You 
wait  at  the  airport  for  two  hours,  attempting  to  contact  someone  who  might  know  what  is  going  on.  but  with  no  luck.  Finally,  a  young  man  casually  walks  in  and 
introduces  himself  as  your  interpreter.  He  offers  no  explanation  for  being  late,  but  his  face  matches  the  picture  that  HQ  gave  you  of  your  interpreter. 

(Rank  the  4  options  from  most  likely  to  do/feel/think  “1",  to  least  likely  to  do/feel/think  “4”,  using  each  number  (1-4)  only  once). 

1.  How  would  you  feel  at  that  moment? 

[l  I  A.  I  would  be  annoyed,  but  I  would  try  to  keep  from  letting  it  show  for  this  first  meeting,  although  I  am  not  sure  I  could  do  that  completely. 


B.  I  would  be  very  angry  and  unable  to  keep  it  in.  We  just  got  done  with  a  long  plane  ride  and  would  be  tired,  and  he  just  shows  up  with  no  explanation. 
He  would  get  yelled  at. 


C.  I  would  definitely  be  frustrated,  but  I  would  be  able  to  suck  it  up  and  keep  my  cool.  I  would  probably  vent  to  one  of  my  Soldiers  later  and  cool  off 
before  I  brought  it  up  to  the  terp. 


[2j~|  D.  I  would  be  tired  and  frustrated  and  would  probably  raise  my  voice  to  him  a  little  right  there  even  though  there  would  probably  be  a  better  way  to 
handle  it. 

(  GonUnue  ) 


Figure  E-3.  Vignette. 


Your  Cross-Cultural  Results  [  Print  ) 


Interpreting  Your  Scores 

Below,  you  will  see  several  scores:  An  overall  cross-cultural  competence  score  and  a  series 
of  scores  on  the  individual  factors  that  are  a  part  of  cross-cultural  competence.  The  scores 
are  ratings,  and  are  based  on  how  your  answers  compared  with  those  provided  by  over  400 
Army  personnel. 

Remember  that  these  scores  reflect  the  answers  you  gave.  If  you  feel  that  they  do  not 
reflect  your  abilities  and  attitudes  then  you  may  want  to  take  some  time  to  think  about  how 
you  answered  certain  questions  and  then  if  you  still  feel  that  these  ratings  do  not  accurately 
describe  your  strengths  and  weaknesses,  ask  a  fellow  Soldier  or  your  superior  officer  for 
their  impressions  on  your  scores.  Regardless,  these  results  are  for  you,  and  are  intended  to 
give  you  some  things  to  think  about  in  terms  of  your  cross-cultural  attitudes  and  abilities 
and  how  you  might  utilize  and  improve  them  to  support  mission  success.  It  is  up  to  you  to 
determine  how  best  to  use  them. 

Your  Overall  Rating: 

Of  the  five  levels  of  cross-cultural  competence: 

PreDompetent  Beginner  Intermediate  Proficient 

Based  on  your  responses,  your  overall  cross-cultural  competence  rating  is:  ;  ITERMEDIATE 

Soldiers  at  the  TERM  EDIATE  level  of  cross-cultural  competence  usually  have  a  basic  awareness  of  the  importance  of  cross-cultural  knowledge  and  interactions  in 
supporting  mission  success.  They  also  are  willing  to  make  efforts  to  engage  with  the  local  populace  and  learn  about  the  culture  in  order  to  do  their  jobs  better.  These 
Soldiers  tend  to  have  basic  relationship-building  and  persuasion  skills  as  well.  Areas  where  these  Soldiers  may  struggle  tend  to  be  in  their  openness  to  extreme  cultural 
differences,  and  in  accuracy  of  their  attempts  to  assume  the  perspectives  of  locals.  They  also  may  struggle  in  adjusting  their  cross-cultural  perceptions  or  attitudes 
once  they  are  formed.  Intermediate  level  leaders  take  responsibility  for  the  cultural  interactions  of  his/her  Soldiers,  and  provide  effective  guidance  on  how  to  avoid 
cross-cultural  incidents. 
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Cultural  Acuity 


Advanced 


Figure  E-4.  Overall  score. 
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Figure  E-5.  Individual  factor  scores. 
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Appendix  F 


C-CAT  Feedback 
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Your  Assessment  Results 


Interpreting  Your  Scores 

Your  assessment  scores  consist  of  an  overall  cross-cultural  competence  rating  and  a  series 
of  ratings  on  the  individual  factors  that  are  a  part  of  cross-cultural  competence. 

Remember  that  these  scores  reflect  the  answers  you  gave.  If  you  feel  that  they  do  not 
reflect  your  abilities  and  attitudes  then  you  may  want  to  take  some  time  to  think  about  how 
you  answered  certain  questions  and  then  if  you  still  feel  that  these  ratings  do  not  accurately 
describe  your  strengths  and  weaknesses,  ask  a  fellow  Soldier  or  your  superior  officer  for 
their  impressions  on  your  scores.  Regardless,  these  results  are  for  you,  and  are  intended  to 
give  you  some  things  to  think  about  in  terms  of  your  cross-cultural  attitudes  and  abilities 
and  how  you  might  utilize  and  improve  them  to  support  mission  success.  It  is  up  to  you  to 
determine  how  best  to  use  them. 


Cultur  al  Acuity 
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Your  Overall  Rating: 

Based  on  your  responses,  your  overall  cross-cultural  competence  rating  is: 
PreCompetent  Beginner  Intermediate  Proficient  Advanced 


Soldiers  at  the  PRECOMPETENT  level  of  cross-cultural  competence  often  do  not  recognize  the 
mission-value  in  interacting  with  locals  beyond  what  is  ordered.  During  deployments,  they  tend 
to  actively  avoid  contact  with  the  local  populace  whenever  possible,  especially  during  free  time. 
Soldiers  at  this  level  may  also  get  frustrated  easily  during  required  interactions,  and  may  have 
little  desire  to  hide  this  frustration.  They  are  often  uncomfortable  with  the  uncertainty  involved 
with  cross-cultural  interactions  and  have  little  regard  for  the  customs  of  the  countries  they  are 
operating  in. 

Soldiers  at  the  BEGINNER  level  of  cross-cultural  competence  will  have  some  initial  knowledge 
of  the  customs  of  the  area  they  will  be  operating  in,  and  they  will  use  this  basic  list  of  rules  and 
taboos  to  guide  their  interactions.  These  Soldiers  all  have  at  least  a  minimum  willingness  to 
engage  with  the  local  populace  in  order  to  support  the  mission  but  they  typically  do  not 
recognize  the  benefits  that  cross-cultural  interactions  can  offer  to  mission  success.  Beginners 
often  view  all  people  as  the  same  and,  therefore,  they  will  have  trouble  understanding  what 
locals  may  really  be  thinking  or  feeling.  They  may  have  basic  interpersonal  skills,  but  without 
an  understanding  of  their  own  cultural  influences,  they  may  struggle  in  using  these  skills  in 
cross-cultural  settings. 

Soldiers  at  the  INTERMEDIATE  level  of  cross-cultural  competence  usually  have  a  basic 
awareness  of  the  importance  of  cross-cultural  knowledge  and  interactions  in  supporting  mission 
success.  They  also  are  willing  to  make  efforts  to  engage  with  foreign  nationals  and  learn  about 
the  culture  in  order  to  do  their  jobs  better.  These  Soldiers  tend  to  have  basic  relationship¬ 
building  and  persuasion  skills  as  well.  Areas  where  these  Soldiers  may  struggle  tend  to  be  in 
their  openness  to  extreme  cultural  differences,  and  in  accuracy  of  their  attempts  to  assume  the 
perspectives  of  foreign  nationals.  They  also  may  struggle  in  adjusting  their  cross-cultural 
perceptions  or  attitudes  once  they  are  formed.  Intermediate  level  leaders  take  responsibility  for 
the  cultural  interactions  of  his/her  Soldiers,  and  provide  effective  guidance  on  how  to  avoid 
cross-cultural  incidents. 

Soldiers  at  the  PROFICIENT  level  of  cross-cultural  competence  are  skilled  at  controlling  their 
emotions  during  tense  or  uncertain  cross-cultural  encounters.  They  also  understand  that  their 
interactions  can  have  a  broad  and  long-term  impact  on  mission  success,  especially  in  deployment 
settings.  They  tend  to  remain  flexible  in  their  actions  with,  and  attitudes  toward  foreign 
populations,  allowing  them  to  adjust  their  perceptions  when  new  information  is  available.  These 
Soldiers  also  recognize  their  own  cultural  biases  and  how  they  influence  their  own  perceptions. 
The  culturally  proficient  Soldier  still  has  room  for  development,  however.  While  they  have 
generally  decent  abilities  to  assume  the  perspectives  of  others,  and  assess  the  long-term  effects  of 
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their  actions  on  the  cultural  aspects  of  the  mission,  the  depth  and  accuracy  of  their  understanding 
can  be  improved  by  including  cultural  elements  in  their  assessments. 

Soldiers  at  the  ADVANCED  level  of  cross-cultural  competence  can  adjust  to  new  cultural 
environments  easily,  with  little  advance  preparation  required.  They  are  ultimately  comfortable 
with  the  uncertainty  of  new  situations  and  people,  and  are  highly  skilled  at  both  assessing  the 
cultural  terrain  and  using  their  assessments  to  successfully  control  the  course  of  all  their  cross- 
cultural  encounters.  They  have  a  clear  awareness  of  their  own  cultural  biases  as  well  as  a  strong 
ability  to  predict  both  the  short-  and  long-term  effects  of  their  cultural  interactions  on  mission 
objectives. 


Individual  Factor  Scores 


Factor 

Low 

Moderate 

Hi 

gh 

Cultural  Acuity 

X 

Relationship  Orientation 

X 

Interpersonal  Skills 

X 

Cultural  Relativism 

X 

Cultural  Interest 

X 

1.  Cultural  Acuity 

What  it  is:  Cultural  Acuity  is  the  ability  to  fonn  accurate  cross-cultural  understandings  and 
assessments  of:  encounters  with  locals,  the  perspectives  of  others,  and  the  impact  of  your  actions 
on  longer-term  mission  goals. 

Your  Score:  LOW 

Why  Cultural  Acuity  is  Important: 

The  better  you  can  understand  what  other  people  are  thinking  and  feeling,  the  more  you  can 
understand  their  motivations  and  ensure  that  the  actions  you  take  with  them  achieve  the  desired 
results.  This  puts  you  in  a  much  better  position  to  skillfully  persuade,  manipulate  and  negotiate 
with  individuals  from  other  cultures.  Also,  the  better  you  can  predict  the  longer  term  effects  of 
your  actions,  the  more  effective  you  will  be  in  achieving  mission  goals  in  the  long  run.  If  you 
are  also  able  to  accurately  assess  and  understand  complex  situations  and  social  dynamics,  you 
can  more  appropriately  respond  to  unfolding  situations  to  dictate  the  course  of  cross-cultural 
encounters  to  your  advantage. 
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What  it  looks  like: 

Soldiers  who  are  high  in  Cultural  Acuity  recognize  that  their  own  perspectives  may  be 
completely  different  and  not  necessarily  more  accurate  than  those  of  others.  They  understand  that 
cultural  differences  influence  everyone’s  perceptions,  which  makes  them  better  able  to 
understand  other  people’s  actions  and  motivations.  They  are  also  able  to  see  beyond  the 
immediate  results  of  their  actions  to  long-term  consequences.  This  helps  guide  their  actions  more 
wisely  to  support  overall  mission  goals.  They  can  also  make  sense  of  often  confusing  and  tense 
cross-cultural  encounters.  They  pick  up  on  critical  cues  as  to  what  people  are  really  thinking 
beyond  what  they  are  saying. 

Soldiers  who  are  low  in  Cultural  Acuity  may  find  it  hard  to  understand  others  people’s 
perspectives.  They  tend  to  interpret  other  people’s  actions  and  interactions  with  their  own  point 
of  view,  making  it  hard  to  get  an  accurate  sense  of  what  others  are  thinking  and  why  they  are 
acting  in  certain  ways. 

These  Soldiers  may  also  find  it  difficult  to  pick  up  on  subtle  details  that  reveal  how  someone  is 
feeling  or  what  their  next  actions  might  be.  It  takes  practice  and  skill  to  see  past  what  someone 
says  with  words  -to  read  their  body  language,  (tone  of  voice,  facial  expressions,  how  they  are 
carrying  themselves)  which  can  reveal  how  they  truly  feel.  It  may  also  be  difficult  to  pick  up  on 
the  many  subtle  cultural  cues  in  deployment  settings. 

How  to  improve  your  Cultural  Acuity: 

•  Know  the  Culture:  When  interacting  with  foreign  nationals,  consider  how  each  person’s 
cultural  background,  lifestyle  and  situation,  family  dynamics,  etc.  might  affect  how  they 
are  thinking  and  why  they  might  behave  or  act  in  a  certain  way. 

•  Learn  from  your  successes  and  mistakes:  After  an  interaction  that  went  either  well  or 
poorly,  think  about  where  you  seemed  to  either  go  right  or  wrong  in  assessing  the 
person’s  intent  or  actions.  Ask  your  interpreter  for  their  view  on  what  happened,  and 
compare  the  experience  with  others  you  have  had. 

•  Ask  questions:  Ask  fellow  Soldiers  who  are  known  for  understanding  the  local  people 
well  and  making  sense  of  complicated  cultural  situations  about  what  they  look  for  during 
interactions. 

•  Observe  details:  Practice  looking  for  subtle  cues  in  situations  and  interactions,  and  see 
how  they  match  or  do  not  match  how  things  play  out. 

•  Always  consider  bigger  picture:  Get  in  the  habit  of  asking  yourself  how  any  action  you 
may  take  will  impact  the  perceptions  that  the  local  populace  has  of  US  forces  before  you 
take  the  action. 

•  Avoid  rigid,  broad  assumptions:  View  each  situation  as  somewhat  unique;  there  may  be 
similarities  to  other  situations,  but  don’t  assume  that  people  will  act  exactly  as  expected. 
Keep  an  open  mind  as  you  observe  and  interpret  cues. 
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